How Python and Ruby Are Implemented in C

Python and Ruby are beloved for their human-friendly syntax and dynamic flexibility, yet their primary reference implementations—CPython and CRuby (MRI)—are built entirely in C. This article explores how these runtimes bridge the gap between dynamic code and hardware execution by examining their virtual machine architectures, dynamic type systems, bytecode compilation pipelines, and underlying memory management models.

Source Code to Bytecode Compilation

Neither Python nor Ruby executes raw source text directly. Instead, both runtimes use standard compiler pipelines implemented in C to transform high-level scripts into compact, intermediate instructions called bytecode.

  1. Tokenization and Parsing: The runtime reads the source file, converts characters into tokens using lexical analyzers, and builds an Abstract Syntax Tree (AST).
  2. Bytecode Generation: A C-based compiler traverses the AST and translates it into an array of bytecode instructions designed specifically for the runtime’s internal virtual machine. In CPython, this produces .pyc files or in-memory code objects (PyCodeObject). In CRuby, this targets the YARV (Yet Another Ruby VM) instruction set.

The Virtual Machine Evaluation Loop

At the heart of both CPython and CRuby lies a stack-based virtual machine written in C. The core engine is conceptually simple: a massive, highly optimized dispatch loop (traditionally an infinite loop containing a large switch statement or computed goto table).

  • CPython (ceval.c): The function _PyEval_EvalFrameDefault fetches bytecode opcodes one by one. If it encounters BINARY_ADD, the C code pops two operands off Python's value stack, calls the internal C addition logic, and pushes the result back onto the stack.
  • CRuby (vm_eval.c / insns.def): YARV operates similarly, reading dynamic instructions and dispatching tasks to dedicated C routines that manipulate local frames and call stacks.

Because hardware processors cannot read Python or Ruby instructions directly, the C execution loop acts as an intermediary processor simulated entirely in software.

Dynamic Typing Through C Structs

C is statically typed, but Python and Ruby are dynamically typed. To handle dynamic values, both runtimes represent every user-facing object using C structures and pointer tagging.

  • CPython's PyObject: Every Python object begins with a base C struct called PyObject. This struct contains two critical fields: a reference count (for memory management) and a pointer to a PyTypeObject. The type object holds function pointers to the appropriate behaviors for that type (such as how to add, stringify, or hash the object). When executing a + b, CPython checks the type descriptor in a's C struct to invoke the corresponding C function.
  • Ruby's VALUE and RBasic: CRuby represents all objects as a single C type called VALUE, which is an integer pointer width. Small integers (Fixnums), nil, true, and symbols are stored immediately inside the pointer itself using bit-flagging (tagged pointers) to avoid heap allocation. Complex objects point to an RBasic struct that contains operational flags and a pointer to the object's class (RClass).

Memory Management and Garbage Collection

High-level developers do not manually manage memory via malloc and free; instead, the C runtimes manage heap allocations automatically.

  • Reference Counting and Cyclic GC (CPython): CPython primarily tracks object lifetimes using reference counts embedded in PyObject. When a reference count drops to zero, the runtime immediately calls C's free() (or its custom allocator, PyObject_Free). To detect reference cycles (e.g., Object A pointing to Object B and vice versa), a secondary cycle-detecting garbage collector periodically traverses object graphs in C.
  • Mark-and-Sweep (CRuby): CRuby relies on a tracing garbage collector. When memory needs to be reclaimed, the runtime halts execution to "mark" all live objects reachable from global tables, the C call stack, and local execution frames. Afterward, it sweeps through the object space, freeing memory occupied by unflagged objects.

Native System Access and C Extensions

Implementing runtimes in C grants high-level languages unrestricted access to operating system system calls (POSIX, Windows API) and hardware primitives. Both Python and Ruby expose this interface outward, allowing developers to write native extension modules in C. When performance bottlenecks arise, high-level code can seamlessly call compiled C functions directly, maintaining the rapid development speed of dynamic scripting with the execution performance of bare-metal C.