PyVarObject in CPython: The ob_size Field Explained

In CPython, memory management and object representation rely on fundamental C structures defined in the Python C API. While fixed-size data structures derive from the baseline PyObject, variable-length entities—such as lists, tuples, and integers—are built upon PyVarObject. This article explains the internal composition of PyVarObject and focuses on ob_size, the specific field it introduces to manage dynamic sequence lengths and memory allocation efficiently.

The Foundation: PyObject vs. PyVarObject

Every object in CPython shares a common foundation defined by the PyObject structure. For fixed-size objects (such as floats), this structure contains only two members, combined under the PyObject_HEAD macro:

• ob_refcnt: A Py_ssize_t integer tracking the reference count for garbage collection.
• ob_type: A pointer to the object's type object (struct _typeobject *).

Variable-length objects require awareness of their item count. To handle this, CPython defines PyVarObject, which incorporates PyObject_VAR_HEAD. This macro includes all fields from PyObject and appends a single new member: ob_size.

The Additional Field: ob_size

The additional field introduced by PyVarObject is:

Py_ssize_t ob_size;

ob_size is a signed, system-dependent integer type (Py_ssize_t, typically a 64-bit signed integer on modern 64-bit systems). Its primary function is to store the number of items or units contained within a variable-length object.

Purpose and Behavior of ob_size

1. O(1) Length Retrieval: Because ob_size stores the length directly in the object header, Python's built-in len() function runs in \(O(1)\) constant time for built-in container types. The interpreter reads this field directly rather than iterating over the contents.
2. Access via Macros: The CPython C API exposes the Py_SIZE(ob) macro to read and set the ob_size field safely.
3. Signed Semantics: Because ob_size is signed, CPython can use negative values in specific internal implementations. For example, in PyLongObject (Python's arbitrary-precision integer), the absolute value of ob_size indicates the number of digits stored, while the sign of ob_size denotes whether the integer itself is positive or negative. A value of zero represents the integer 0.

Common Types Utilizing PyVarObject

Several core Python types use PyVarObject as their underlying structure:

• PyTupleObject: Immutable sequence whose ob_size indicates the exact number of elements stored.
• PyListObject: Dynamic array where ob_size represents the number of active items visible to Python, distinct from the total allocated memory capacity (allocated).
• PyBytesObject: Fixed byte sequence where ob_size specifies the exact byte count.
• PyLongObject: Arbitrary-precision integer where ob_size tracks the number of internal digit chunks and sign.

By introducing ob_size, PyVarObject gives CPython a standardized, highly performant mechanism to handle dynamic-length entities across the interpreter.