How to Use IAsyncEnumerable and await foreach in C

Asynchronous streams in C# provide a mechanism for pulling data asynchronously on demand using the IAsyncEnumerable<T> interface and the await foreach statement. Introduced in C# 8.0, this feature bridges the gap between asynchronous operations (Task<T>) and iterator patterns (IEnumerable<T>), allowing applications to stream data elements as they become available without loading entire datasets into memory or blocking execution threads.

The Problem with Traditional Approaches

Before IAsyncEnumerable<T>, developers faced two suboptimal choices when retrieving collections asynchronously:

  1. Task<IEnumerable<T>>: The entire dataset must be retrieved and buffered in memory before the caller receives anything. This increases latency for large data sets and causes high memory spikes.
  2. IEnumerable<Task<T>>: The collection must already exist in memory, even if resolving each individual item requires an asynchronous call.

Asynchronous streams resolve this by allowing each item to be fetched, yielded, and consumed individually as an asynchronous operation.

Declaring an Asynchronous Stream

An asynchronous stream is defined by returning IAsyncEnumerable<T> and combining the async modifier with the yield return statement. Within the method, you can freely use the await keyword.

public async IAsyncEnumerable<int> GenerateNumbersAsync()
{
    for (int i = 1; i <= 5; i++)
    {
        await Task.Delay(500); // Simulate asynchronous I/O work
        yield return i;
    }
}

In this implementation, the method yields each integer directly to the caller after waiting 500 milliseconds, without generating the subsequent numbers in advance.

Consuming Streams with await foreach

To consume an IAsyncEnumerable<T>, C# provides the await foreach loop. It asynchronously waits for the next element via the underlying IAsyncEnumerator<T>.MoveNextAsync() method:

await foreach (var number in GenerateNumbersAsync())
{
    Console.WriteLine(number);
}

The execution transitions back and forth:

  1. The consumer invokes MoveNextAsync().
  2. The producer executes until it hits yield return and pauses.
  3. The consumer processes the item.
  4. The cycle repeats until the producer finishes.

Handling Cancellation and Context

Real-world scenarios require handling cancellation and managing execution context.

Cancellation Tokens

To pass a cancellation token to an async stream, decorate the token parameter with the [EnumeratorCancellation] attribute inside the producer method:

using System.Runtime.CompilerServices;
using System.Threading;

public async IAsyncEnumerable<string> ReadLogsAsync([EnumeratorCancellation] CancellationToken cancellationToken = default)
{
    while (!cancellationToken.IsCancellationRequested)
    {
        string? logEntry = await FetchNextLogAsync(cancellationToken);
        if (logEntry == null) break;

        yield return logEntry;
    }
}

When consuming the stream, bind your token using the WithCancellation extension method:

var cts = new CancellationTokenSource();
await foreach (var log in ReadLogsAsync().WithCancellation(cts.Token))
{
    Console.WriteLine(log);
}

Configuring Await

In library code or non-UI threads, configure context capturing using ConfigureAwait:

await foreach (var item in stream.ConfigureAwait(false))
{
    // Executes without capturing synchronization context
}

When to Use Asynchronous Streams

  • Streaming I/O: Reading large files, database queries via Object-Relational Mappers (such as Entity Framework Core), or network sockets line-by-line.
  • Real-Time Data Feeds: Processing incoming messages from message brokers (e.g., RabbitMQ, Kafka) or webhooks.
  • Paging APIs: Fetching paginated REST APIs where subsequent pages are retrieved only when requested by the consumer.