C Stack Overflow vs Python RecursionError Explained

While both errors arise from deeply nested execution, they represent entirely different failure boundaries within runtime environments. A Python RecursionError is a preemptive, software-enforced limit managed directly by the CPython interpreter to protect system stability, whereas a C-level stack overflow is a hard hardware and operating system fault caused by exhausting the actual physical memory allocated to a thread's call stack. Understanding the difference requires looking at how Python abstracts execution frames versus how native C code utilizes system memory.

Python RecursionError: A Software Counter Safeguard

Python executes code by allocating frame objects on the heap rather than relying strictly on the native operating system stack for Python-to-Python function calls. Because Python frames are heap-allocated, pure Python recursion would theoretically continue consuming system RAM until the machine ran out of memory, or until native C functions within the interpreter consumed the OS stack.

To prevent silent crashes and unconstrained resource consumption, Python implements a logical depth limit:

• Trigger Mechanism: CPython tracks recursion depth using an internal counter (PyThreadState.recursion_depth). Every time a Python function is entered, this counter increments; when the function returns, it decrements.
• The Threshold: If this counter reaches the threshold defined by sys.getrecursionlimit() (typically set to 1,000 by default), the interpreter halts execution and raises a RecursionError (a subclass of BuiltinError).
• Consequence: Because this is a controlled runtime exception, it can be captured using a standard try...except RecursionError block. The Python process remains healthy and stable.

C-Level Stack Overflow: Physical Memory Depletion

Unlike Python's logical frame counting, C functions allocate local variables, return addresses, and execution contexts directly on a contiguous block of virtual memory known as the native call stack. This stack is allocated by the operating system when a thread is spawned (typically ranging from 1 MB to 8 MB).

• Trigger Mechanism: A native stack overflow occurs when native execution pushes the stack pointer past the boundary of allocated stack memory. The operating system places a "guard page" at the end of the stack region—an unmapped page of memory.
• The Threshold: The trigger is not a function count, but raw bytes. Declaring massive local arrays on the stack (e.g., int buffer[1000000];) or recursing thousands of times through native C functions moves the stack pointer onto the guard page.
• Consequence: When the CPU attempts to write to or read from the guard page, the Memory Management Unit (MMU) generates a hardware fault. The operating system intercepts this as an illegal memory access and sends a signal (such as SIGSEGV on Unix-like systems or a structured exception EXCEPTION_STACK_OVERFLOW on Windows). The process terminates immediately with a segmentation fault. Standard Python exception handlers cannot catch this event.

Key Differences

When Python Encounters a C Stack Overflow

Although Python's recursion limit is designed to prevent native stack exhaustion, a C-level stack overflow can still occur in Python applications under specific conditions:

1. Artificially High Recursion Limits: Increasing sys.setrecursionlimit() to a very high number (such as 100,000) permits deep call stacks. Because each Python call still involves internal C-level dispatch functions inside the interpreter, native stack space is gradually consumed. Eventually, the native OS stack runs out before the Python limit is reached, causing a segmentation fault.
2. C Extensions and Native Libraries: Deep recursion occurring inside C extension modules (e.g., NumPy, Cython, or custom C libraries) bypasses Python's internal frame counter entirely. If native functions call each other recursively without sufficient stack space, a hardware crash occurs instantly.
3. Deep C-Level Object Nesting: Evaluating or deallocating deeply nested objects (such as a dictionary containing thousands of nested lists) can cause recursive calls to internal C functions like Py_DECREF. If the C deallocator recurses too deeply, it triggers a C-level stack overflow outside the purview of RecursionError.