Rate Limiting API Requests with asyncio.Semaphore

This article explains how Python's asyncio.Semaphore functions as an essential concurrency control mechanism for managing outbound API requests. You will learn the mechanics behind the semaphore, why it is necessary to prevent API rate limit violations and server overload, and how to implement it cleanly using an asynchronous context manager to bound concurrent network tasks.

What is an asyncio.Semaphore?

An asyncio.Semaphore is a synchronization primitive built on an internal counter. When you instantiate a semaphore, you assign it an integer value representing the maximum number of tasks allowed to access a shared resource simultaneously:

import asyncio

sem = asyncio.Semaphore(10)  # Allows up to 10 concurrent operations

Each time a coroutine calls acquire() (or enters an async with sem: block), the counter decreases by one. If the counter reaches zero, any subsequent coroutine attempting to acquire the semaphore suspends execution without blocking the event loop. When a running coroutine completes its work and calls release() (or exits the context block), the counter increments by one, immediately allowing the next waiting coroutine in line to proceed.

The Role of Semaphore in API Rate Limiting

Asynchronous HTTP libraries like aiohttp or httpx allow Python programs to dispatch thousands of requests in seconds. However, sending unconstrained traffic causes critical issues:

asyncio.Semaphore prevents these issues by enforcing a strict ceiling on the number of in-flight requests. While it controls concurrency (simultaneous connections) rather than a strict rate-over-time (such as requests per minute), concurrency capping is often the first and most critical defense against overwhelming APIs.

Implementing asyncio.Semaphore

The safest way to use asyncio.Semaphore is via the asynchronous context manager (async with), which guarantees that the acquired permit is automatically released even if an exception occurs during the HTTP request.

import asyncio
import httpx

MAX_CONCURRENT_REQUESTS = 5
semaphore = asyncio.Semaphore(MAX_CONCURRENT_REQUESTS)

async def fetch_url(client: httpx.AsyncClient, url: str) -> int:
    async with semaphore:
        response = await client.get(url)
        return response.status_code

async def main():
    urls = [f"https://httpbin.org/get?id={i}" for i in range(50)]
    
    async to_run = []
    async with httpx.AsyncClient() as client:
        tasks = [fetch_url(client, url) for url in urls]
        results = await asyncio.gather(*tasks)
        print(f"Completed {len(results)} requests.")

if __name__ == "__main__":
    asyncio.run(main())

In this implementation, even though 50 tasks are scheduled simultaneously with asyncio.gather, only five tasks can execute their client.get() block at any given millisecond. The remaining 45 tasks wait cooperatively in the event loop queue until slots become available.

Concurrency vs. Throughput Rate Limits

It is important to distinguish between concurrency limits and frequency limits:

  1. Concurrency (Handled by Semaphore): "No more than 10 requests active at the same moment."
  2. Frequency (Handled by Token Buckets/Leaky Buckets): "No more than 100 requests per 60 seconds."

If an API explicitly restricts calls per unit of time, a semaphore alone may still violate limits if the target server responds quickly (for instance, completing 10 parallel requests in 50 milliseconds allows 200 requests per second). In such scenarios, asyncio.Semaphore should be paired with a delay mechanism (such as asyncio.sleep) or a specialized rate-limiting algorithm like a token bucket to throttle both concurrency and velocity.