C++ Atomic Wait and Notify: Replacing Busy Waiting

This article provides an overview of the atomic wait and notify primitives introduced in C++20, detailing their mechanics, performance implications, and usage. You will learn why traditional busy waiting harms system performance, how functions like wait(), notify_one(), and notify_all() resolve these issues at the hardware and operating system levels, and how to effectively implement them in modern concurrent C++ applications.

The Problem with Busy Waiting

In multithreaded programming, threads frequently need to wait for a state change produced by another thread. Historically, developers often used busy waiting (also known as spinning) to wait for an atomic variable to change:

std::atomic<bool> ready{false};

// Waiting thread
while (!ready.load(std::memory_order_acquire)) {
    // Spin: consumes 100% of a CPU core
}

While busy waiting offers the lowest possible latency when the wait duration is predictable and extremely short, it introduces severe drawbacks:

  • Excessive CPU Consumption: The spinning thread continuously executes instructions, pegging a CPU core at 100% utilization.
  • Power and Thermal Throttling: Constant core utilization generates heat and depletes battery life on mobile and embedded devices.
  • Resource Starvation: Spinning threads consume CPU cycles that could otherwise be scheduled for threads doing actual work, potentially causing priority inversion.
  • Bus Contention: Continuous atomic loads can flood the memory bus and invalidate cache lines, slowing down other cores.

Alternative solutions, such as std::this_thread::yield() or std::this_thread::sleep_for(), either fail to release CPU resources efficiently or introduce unacceptable, non-deterministic latency.

What Are Atomic Wait and Notify Primitives?

Introduced in the C++20 standard, wait and notify primitives are member functions of std::atomic and std::atomic_flag. They provide a mechanism to block a thread until an atomic variable's value changes, bridging the gap between low-overhead atomics and operating-system-level thread suspension.

The primary member functions added to std::atomic<T> include:

  • void wait(T old, std::memory_order order = std::memory_order_seq_cst) const noexcept;
    Compares the current value of the atomic variable with old. If they are equal, the thread is blocked until a notification is received or a spurious wakeup occurs. If the value has already changed, the function returns immediately without blocking.
  • void notify_one() noexcept;
    Unblocks at least one thread currently waiting on the atomic variable via wait().
  • void notify_all() noexcept;
    Unblocks all threads currently waiting on the atomic variable via wait().

Under the hood, these primitives map directly to efficient platform-specific kernel mechanisms, such as futex (Fast Userspace Mutex) on Linux or WaitOnAddress on Windows. When blocked, the thread is put into a sleep state by the OS scheduler, releasing the CPU core entirely.

How They Replace Busy Waiting

Atomic wait and notify primitives completely eliminate the CPU spin loop while retaining a simple, atomic-centric interface.

Instead of polling a condition in a loop, a consumer thread calls wait(), passing the value it expects to change:

#include <atomic>
#include <thread>

std::atomic<bool> ready{false};

void worker() {
    // Blocks efficiently if ready == false; consumes zero CPU while waiting
    ready.wait(false, std::memory_order_acquire);
    
    // Proceed with work after notification
}

void publisher() {
    // Perform setup...
    ready.store(true, std::memory_order_release);
    ready.notify_one(); // Wakes up the waiting worker thread
}

Because spurious wakeups are possible at the operating-system level, robust implementations typically place the wait() call inside a lightweight loop to verify the state condition:

while (!ready.load(std::memory_order_acquire)) {
    ready.wait(false, std::memory_order_relaxed);
}

Even though a loop is used, the thread sleeps on ready.wait() instead of continuously polling the CPU.

Advantages Over Traditional Mutexes and Condition Variables

Prior to C++20, the standard way to avoid busy waiting without atomics was using std::condition_variable paired with a std::mutex. While effective, this approach introduces noticeable overhead:

  1. No Mutex Required: std::atomic::wait does not require locking or unlocking a mutex, eliminating mutex-related overhead and potential lock contention.
  2. Memory Footprint: A std::condition_variable accompanied by a std::unique_lock and std::mutex consumes substantially more memory than a single scalar std::atomic<T>.
  3. Simpler Lifetimes: State synchronization using atomic primitives avoids deadlock risks associated with improper mutex lock ordering.

Atomic wait and notify primitives combine the minimal overhead of lock-free programming with the resource efficiency of operating-system-level thread suspension, making them the preferred modern C++ solution for synchronization scenarios that previously relied on busy waiting.