Optimizing Python Memory with slots

In Python, standard class instances store their attributes in a dynamic dictionary named __dict__, which provides flexibility but incurs a significant memory overhead. By declaring __slots__, developers can bypass this default dictionary mechanism and allocate a fixed amount of space for a predefined set of attributes. This optimization drastically reduces the memory footprint of individual objects, improves attribute access speed, and prevents the dynamic addition of unapproved attributes, making it particularly valuable when instantiating millions of small objects.

The Default Behavior: The __dict__ Overhead

By default, Python classes are designed to be dynamic. You can add, modify, or remove attributes on an instance at runtime. To facilitate this flexibility, every instance creates an internal dictionary:

class StandardPoint:
    def __init__(self, x, y):
        self.x = x
        self.y = y

p = StandardPoint(1, 2)
print(p.__dict__)  # Output: {'x': 1, 'y': 2}

A Python dictionary is a hash table. To avoid hash collisions and allow fast key-value lookups, dictionaries over-allocate memory. In addition to the memory consumed by the dictionary structure itself, the instance pointer to that dictionary adds another layer of overhead. When creating millions of instances of StandardPoint, this overhead quickly adds up to gigabytes of unnecessary RAM usage.

How __slots__ Optimizes Memory

When you define __slots__ inside a class, you tell Python not to use a dynamic dictionary for instance attributes. Instead, Python allocates a fixed-size array of references, similar to a struct in C:

class SlottedPoint:
    __slots__ = ('x', 'y')

    def __init__(self, x, y):
        self.x = x
        self.y = y

By assigning a tuple of attribute names to __slots__, several key optimizations occur:

1. Elimination of __dict__: The instance no longer creates or maintains a per-instance dictionary, saving roughly 100 to 150 bytes per object on 64-bit systems.
2. Compact Memory Layout: Object attributes are stored as sequential pointers at fixed offsets directly in the object structure.
3. Descriptor-Based Access: Python automatically generates descriptors for each slotted attribute, allowing direct memory offset lookups instead of hashing string keys.

Quantifying the Memory Savings

The difference in memory consumption between standard instances and slotted instances is substantial when scaled across large datasets.

import sys

p_standard = StandardPoint(10, 20)
p_slotted = SlottedPoint(10, 20)

# Memory of instance itself
print(sys.getsizeof(p_standard))        # ~48-56 bytes
print(sys.getsizeof(p_standard.__dict__)) # ~104-112 bytes
print(sys.getsizeof(p_slotted))         # ~48-56 bytes (with no __dict__)

Because sys.getsizeof() does not recursively count nested structures like __dict__, the true cost of p_standard is the sum of the instance and its dictionary (typically 150+ bytes), whereas p_slotted remains fixed around 48 to 56 bytes. In production environments involving millions of records, implementing __slots__ typically reduces overall process memory by 40% to 70%.

Secondary Benefit: Faster Attribute Access

While primarily a memory optimization technique, __slots__ also provides a measurable speed improvement. Accessing a slotted attribute does not require hashing a string name and searching a hash table. Instead, the runtime reads directly from a predetermined memory address, resulting in roughly 15% to 25% faster attribute reads and writes.

Trade-offs and Limitations

While powerful, __slots__ introduces constraints that must be considered:

• Fixed Attributes: You cannot dynamically add new attributes to an instance at runtime. Attempting to set an undefined attribute raises an AttributeError.
• Inheritance Rules: A subclass does not automatically inherit slotted behavior. If a child class inherits from a slotted parent without defining its own __slots__, Python creates a __dict__ for the child, negating the memory benefits.
• Weak References: Slotted classes cannot be weakly referenced using Python's weakref module unless '__weakref__' is explicitly included in the __slots__ definition.
• Multiple Inheritance: Multiple inheritance with multiple slotted base classes is strictly restricted if more than one base class defines non-empty __slots__.

Using __slots__ is an effective pattern in Python when building data-heavy classes, parsers, or simulation engines where memory constraints are critical.