How Python Handles Signals Across Native Threads

Python restricts the execution of user-defined signal handlers exclusively to the main interpreter thread, even though underlying operating systems can deliver asynchronous signals to any native thread. When an operating system signal occurs, CPython relies on a combination of a low-level C-level handler, internal execution flags, and the Global Interpreter Lock (GIL) to catch the signal on whichever native thread receives it, defer execution, and run the Python-level callback safely on the main thread.

The Low-Level C Signal Handler

At the OS level (using POSIX threads or Windows equivalents), signals sent to a process can be intercepted by any native thread that does not currently mask that signal. When you register a handler using Python's signal.signal() function, CPython does not directly attach your Python callable to the OS signal table. Instead, CPython registers a generic low-level C function (traditionally signal_handler in Modules/signalmodule.c) with the operating system via sigaction() or signal().

When an OS signal arrives, the kernel interrupts whichever native OS thread is scheduled to accept it and invokes CPython's C signal handler.

Flagging Pending Signals Without Python Execution

A native thread interrupted by an OS signal might not hold the Global Interpreter Lock (GIL), nor might it possess a valid Python thread state. Executing arbitrary Python bytecode directly inside an OS-level interrupt context is unsafe and would cause memory corruption or deadlocks.

To avoid this, the low-level C handler performs only minimal, asynchronous-safe work:

It marks the signal number as pending by setting a flag in an internal C runtime array (Handlers[signum].tripped = 1).
It sets an evaluation breaker flag (such as _Py_eval_breaker or eval_flags) in the interpreter runtime.
If configured (for example, via signal.set_wakeup_fd()), it writes the signal byte into a non-blocking wakeup file descriptor to interrupt any polling mechanisms like select() or epoll().

Once these flags are updated, the OS interrupt handler completes, and the native thread resumes its normal execution.

Transferring Control to the Main Thread

The Python virtual machine evaluates bytecode inside a central loop (in Python/ceval.c). Periodically, and at every loop iteration, the interpreter checks its evaluation flags to manage thread switching, asynchronous calls, and signals.

When the interpreter detects that the signal breaker flag is set, it invokes internal checking functions:

Background Threads: If a native background thread notices the breaker flag, it ignores Python-level signal processing and continues its tasks (or yields the GIL if a thread switch was requested).
The Main Thread: Only the main thread is authorized to run Python signal handlers. When the main thread holds the GIL and checks the evaluation flags, it inspects the table of tripped signals via PyErr_CheckSignals().

Executing the Python Callback

Upon detecting tripped signals in the main thread, CPython:

Resets the tripped flag for that signal.
Looks up the associated Python callable registered in the signal module.
Invokes the Python callable within the main thread's context, passing the signal number and the current execution frame.
If the handler raises an exception (such as KeyboardInterrupt from SIGINT), that exception is injected directly into the main thread at its current execution point.

Handling Blocked System Calls

If the main thread is blocked inside an interruptible system call (such as time.sleep(), socket reads, or file I/O) when a signal is tripped:

The OS kernel interrupts the system call, returning an EINTR error.
CPython's internal wrappers catch EINTR and immediately invoke PyErr_CheckSignals().
The main thread runs the Python signal handler.
If the handler executes without raising an exception, Python's runtime automatically retries the interrupted system call (as governed by PEP 475), ensuring operations resume smoothly without manual retry loops.