Python Async Iterators, Generators, and Async For

Asynchronous programming in Python extends beyond coroutines to managing streaming data concurrently. This guide explains the mechanics of asynchronous iterators and asynchronous generators, detailing how the async for loop consumes non-blocking data streams. By the end, you will understand how to build and consume custom asynchronous sequences without blocking Python's event loop.

What is an Asynchronous Iterator?

A standard Python iterator implements the __iter__() and __next__() methods. When dealing with I/O-bound operations—such as reading data from a network socket or streaming records from a database—synchronous iterators block execution while waiting for the next item.

An asynchronous iterator solves this by implementing two specific dunder methods:

Here is an example of a custom asynchronous iterator:

import asyncio

class AsyncCounter:
    def __init__(self, limit):
        self.limit = limit
        self.count = 0

    def __aiter__(self):
        return self

    async def __anext__(self):
        if self.count < self.limit:
            await asyncio.sleep(0.5)  # Simulate non-blocking I/O
            self.count += 1
            return self.count
        else:
            raise StopAsyncIteration

What is an Asynchronous Generator?

Writing a class with __aiter__ and __anext__ can be verbose. An asynchronous generator provides a more concise way to create an asynchronous iterator using standard function syntax.

An asynchronous generator is created by placing the yield expression inside an async def function. Unlike normal generators, an async generator can use both await and yield within its body.

import asyncio

async def async_fetch_data(limit):
    for i in range(1, limit + 1):
        await asyncio.sleep(0.5)  # Simulate I/O latency
        yield f"Record {i}"

Calling async_fetch_data() does not execute the function immediately; it returns an asynchronous generator object that implements the async iterator protocol.

How async for Is Used

The async for statement is used to iterate over an asynchronous iterator or generator. Because retrieving the next item involves an asynchronous operation, async for can only be used inside an async def coroutine.

Under the hood, async for repeatedly awaits the __anext__() method of the iterator and terminates cleanly when it catches StopAsyncIteration.

import asyncio

async def main():
    # Consuming the async generator
    print("Streaming records:")
    async for record in async_fetch_data(3):
        print(record)

    # Consuming the custom async iterator class
    print("\nCounting:")
    async for number in AsyncCounter(3):
        print(number)

asyncio.run(main())

Output:

Streaming records:
Record 1
Record 2
Record 3

Counting:
1
2
3

Key Differences Between Synchronous and Asynchronous Iteration

Feature Synchronous Asynchronous
Iterator Methods __iter__(), __next__() __aiter__(), __anext__()
Generator Definition def containing yield async def containing yield
Termination Signal StopIteration StopAsyncIteration
Consumption Syntax for item in iterable: async for item in async_iterable:
Execution Context Anywhere Inside an async def coroutine

When to Use Asynchronous Iterators

Use asynchronous iterators and async for when processing data that arrives in chunks over time, such as: