How Await Yields Control to the Event Loop in Python

In Python's asynchronous programming model, the await keyword pauses the execution of an enclosing coroutine and yields control back to the event loop. This article explains the underlying mechanics of this process, detailing how coroutines operate as state machines, how the __await__ protocol interacts with generator-style yielding, and how the event loop manages task scheduling to achieve cooperative multitasking without blocking the thread.

The Foundation: Coroutines as Generators

Under the hood, Python coroutines created with async def are syntactic and structural evolutions of generators. While regular functions run from start to finish and return a single value, generators and coroutines can pause their execution state—including local variables, instruction pointers, and exception states—and resume later.

When you define an async def function, Python compiles it into a coroutine object. Like a generator, a coroutine object cannot execute on its own; it requires a driver to advance its execution step by step. In asynchronous Python, the event loop acts as this driver.

The __await__ Dunder Method

When Python encounters the expression await obj, it does not immediately halt the entire thread. Instead, it follows these exact steps:

1. Python checks if obj is an awaitable by looking for the __await__() magic method.
2. It calls obj.__await__(), which must return an iterator.
3. Python then iterates through this iterator, propagating whatever values the iterator yields up through the call stack.

Most commonly, the object being awaited is an asyncio.Task or an asyncio.Future.

The Actual Yield Mechanism

The magic of transferring control back to the event loop relies on a standard yield statement hidden deep inside the standard library.

At the lowest level of asyncio, an asyncio.Future implements __await__ roughly like this:

def __await__(self):
    if not self.done():
        self._asyncio_future_blocking = True
        yield self  # Control is yielded here
    return self.result()

When a future is not yet resolved, it yields itself. Because the coroutine executing await delegates to this iterator, the yield bubbles up through any nested coroutine calls until it reaches the task runner inside the event loop.

How the Event Loop Receives Control

1. Execution Steps Forward: The event loop runs a scheduled task by calling its .send(None) method. This starts or resumes the coroutine.
2. Hit the Suspension Point: The coroutine runs synchronously until it reaches an I/O operation, sleep, or another awaited future that is not complete.
3. Yielding Back: The uncompleted future executes yield self. This suspends the execution frame of the coroutine and returns the Future object back to the event loop's task-driving loop.
4. Registration: The event loop sees that the task has yielded a future that is not done. It attaches a callback to that future: future.add_done_callback(loop._wakeup).
5. Loop Continues: Because the coroutine has paused and returned execution back to the loop's caller frame, the event loop is now free to poll the OS for network I/O selectors, execute scheduled timers, or step another ready task forward.

Resuming the Coroutine

Once the background operation finishes (such as the operating system notifying Python that socket data has arrived, or a timer expiring):

1. The low-level callback fires and marks the Future as completed with a result or exception.
2. The event loop moves the associated task back into its "ready" queue.
3. On a subsequent iteration of the loop, the task is driven forward again using coroutine.send(result).
4. The coroutine receives the result directly at the point of the original await expression and continues execution until it completes or hits another await.