Task Cancellation and Cleanup in Python asyncio

In Python's asyncio, task cancellation is an event-driven mechanism that interrupts running coroutines by injecting an asyncio.CancelledError at their current await point. Managing this lifecycle properly requires using standard Python control structures, such as try...finally blocks and asynchronous context managers, to ensure resources like sockets, file descriptors, and database connections are gracefully released when a task is terminated prematurely.

How Task Cancellation Works

When you want to stop a running background task, you call the cancel() method on the asyncio.Task instance. Calling task.cancel() does not immediately kill the execution thread; instead, it schedules an asyncio.CancelledError exception to be raised inside the coroutine the next time it pauses at an await statement.

import asyncio

async def worker():
    try:
        print("Worker running...")
        await asyncio.sleep(5)
    except asyncio.CancelledError:
        print("Cancellation requested!")
        raise  # Must propagate unless explicitly suppressing

async def main():
    task = asyncio.create_task(worker())
    await asyncio.sleep(1)
    task.cancel()
    
    try:
        await task
    except asyncio.CancelledError:
        print("Task was successfully cancelled.")

asyncio.run(main())

If the coroutine does not catch asyncio.CancelledError, the task terminates, and the exception is stored on the task. Awaiting a cancelled task re-raises the asyncio.CancelledError in the caller's scope.

Resource Cleanup with try...finally

The primary way to handle cleanup during task cancellation is the try...finally pattern. Since CancelledError inherits from BaseException (as of Python 3.8), a finally block is guaranteed to execute whether the task completes normally, raises an unhandled error, or gets cancelled.

async def managed_resource_worker():
    resource = await acquire_resource()
    try:
        while True:
            await process_data(resource)
    finally:
        # Guaranteed to run upon cancellation
        await resource.close()

Running Asynchronous Cleanup During Cancellation

If your cleanup logic requires awaiting asynchronous operations (such as closing network sessions or flushing buffers), calling await inside an except asyncio.CancelledError or finally block can cause issues if the task is cancelled again.

To perform asynchronous cleanup safely inside an ongoing cancellation, wrap the cleanup operation in asyncio.shield():

async def safe_cleanup_worker():
    try:
        await asyncio.sleep(10)
    except asyncio.CancelledError:
        # Shield cleanup so it cannot be interrupted by another cancellation
        await asyncio.shield(release_external_lock())
        raise

Structured Concurrency with TaskGroup

Introduced in Python 3.11, asyncio.TaskGroup provides structured concurrency that simplifies task cancellation and cleanup across multiple related tasks. If any task inside a TaskGroup fails, all other tasks in the group are automatically cancelled, and their cleanup blocks are executed before the context manager exits.

async def fetch_data(id):
    await asyncio.sleep(id)
    return {"id": id}

async def run_group():
    try:
        async with asyncio.TaskGroup() as tg:
            task1 = tg.create_task(fetch_data(1))
            task2 = tg.create_task(fetch_data(2))
    except* Exception as eg:
        # Handles any aggregated exceptions raised by tasks
        print(f"Group failed: {eg.exceptions}")

Using TaskGroup guarantees that no orphaned background tasks continue running if a sibling task encounters an error.

Key Rules for Handling Cancellation