Python faulthandler: Debug Low-Level C Crashes
When working with C extensions or native shared libraries in Python,
severe errors often trigger fatal system signals like segmentation
faults that terminate the Python process without leaving a standard
traceback. The built-in faulthandler module bridges this
gap by catching low-level operating system signals and printing the
Python traceback directly to standard error before the interpreter dies.
This guide explains what the faulthandler module
accomplishes, how it aids in diagnosing native C crashes, and how to
implement it effectively.
The Problem with Low-Level C Crashes
Standard Python exceptions rely on the Python runtime remaining intact. When a bug occurs in pure Python, the runtime catches the error and displays a structured traceback indicating the file, line number, and call stack.
However, when an error occurs inside a compiled C extension, a Cython
module, or through ctypes/cffi—such as
dereferencing a null pointer, writing to protected memory, or
encountering a division by zero—the operating system raises a signal
(e.g., SIGSEGV). By default, this immediately terminates
the process. The operating system simply outputs a terse error like
Segmentation fault (core dumped), leaving developers with
no indication of which Python function or line triggered the native
failure.
What faulthandler Accomplishes
The faulthandler module registers signal handlers for
fatal POSIX signals and Windows fatal exceptions. When one of these
events occurs, it accomplishes several critical debugging tasks:
1. Intercepts Fatal System Signals
faulthandler registers handlers for the most common
fatal signals:
SIGSEGV(Segmentation fault)SIGFPE(Floating point exception)SIGABRT(Process abort signal)SIGBUS(Bus error)SIGILL(Illegal instruction)
On Windows, it handles fatal exceptions through Windows Structured Exception Handling (SEH).
2. Dumps the Python Call Stack
Before allowing the process to terminate, faulthandler
translates the internal Python thread state into a human-readable
traceback. It prints this traceback directly to sys.stderr
(or an alternate designated file descriptor). This allows developers to
see the exact line of Python code that invoked the native function
responsible for the crash.
3. Provides Multi-Thread Tracebacks
In concurrent applications, pinpointing which thread provoked the
native fault can be difficult. faulthandler can inspect and
output tracebacks for all active Python threads at the moment of the
crash, providing crucial context when race conditions in native code
cause memory corruption.
4. Diagnoses Freezes and Deadlocks
Beyond signal handling, faulthandler includes a watchdog
mechanism via faulthandler.dump_traceback_later(timeout).
If native code enters an infinite loop, blocks indefinitely on I/O, or
deadlocks while holding the Global Interpreter Lock (GIL), this function
triggers a thread dump after the specified timeout without terminating
the process.
How to Enable faulthandler
The module is part of the Python standard library (Python 3.3+) and can be activated using multiple methods:
Environment Variable
Set the environment variable before running the script:
export PYTHONFAULTHANDLER=1
python app.pyCommand-Line Flag
Pass the -X faulthandler option directly to the Python
interpreter:
python -X faulthandler app.pyIn-Code Activation
Enable signal handling programmatically at application startup:
import faulthandler
# Enable standard fault handling to stderr
faulthandler.enable()
# Optionally redirect output to an open file
with open("crash_report.log", "w") as f:
faulthandler.enable(file=f)Limitations to Consider
While faulthandler is a crucial tool for diagnosing
native crashes, it has specific constraints:
- No C Stack Trace: It reconstructs only the Python
interpreter's stack frames. To inspect the C-level stack frames and
symbols inside the native library itself, tools like
gdb,lldb, or Valgrind are required. - Async-Signal Safety: Because it executes inside an
operating system signal handler,
faulthandlermust remain strictly async-signal-safe. It cannot allocate memory, invoke Python code, or run arbitrary clean-up tasks; it merely reads existing runtime memory structures and writes raw bytes to a file descriptor. - Severe Memory Corruption: If the native bug
corrupts Python's own internal memory structures (such as the thread
state or frame pointers),
faulthandlermay be unable to parse the stack and could abort silently.