How Custom Allocators Prevent Heap Fragmentation in Games

Continuous asset loading in modern game engines introduces severe heap fragmentation when relying on general-purpose system allocators. This article explores how custom memory allocators mitigate fragmentation by segregating asset lifetimes, standardizing allocation sizes, and managing contiguous memory blocks. By replacing non-deterministic heap behavior with specialized strategies like arena, pool, buddy, and handle-based systems, developers can maintain predictable frame times, eliminate out-of-memory crashes, and maximize cache efficiency during runtime streaming.

The Problem: General-Purpose Allocators and Streaming

General-purpose allocators (such as standard malloc or new) are designed for arbitrary allocation sizes and unpredictable lifetimes. In modern game streaming—where textures, audio clips, animation data, and level geometry are constantly loaded and unloaded—this general-purpose model breaks down:

  • External Fragmentation: Free memory becomes divided into tiny, non-contiguous blocks. Over time, an allocation request for a large texture may fail even if total free memory is sufficient.
  • Internal Fragmentation: Allocators pad requests to align with memory boundaries, wasting space across thousands of small assets.
  • Cache Misses and Overhead: Non-contiguous allocations degrade CPU cache locality and introduce non-deterministic traversal times.

Custom allocators solve these issues by imposing domain-specific constraints on how memory is requested and returned.

Lifetime-Based Segregation (Arena and Stack Allocators)

Assets in games naturally fall into distinct lifetime categories: per-frame, per-scene, per-room, or persistent. Standard heaps mix these lifetimes, interleaving short-lived audio instances with long-lived textures.

Arena (linear) and stack allocators group allocations by lifetime:

  • Contiguous Allocation: Memory is allocated sequentially by incrementing an internal offset pointer within a pre-allocated buffer.
  • Batch Deallocation: Individual assets are not freed individually. Instead, the entire arena or stack frame is cleared at once (e.g., when transitioning between levels or unloading a streaming cell) by resetting the offset pointer to zero.
  • Fragmentation Elimination: Because holes are never carved out of the middle of the buffer, external fragmentation is physically impossible within that memory block.

Uniform Sizing (Pool and Free-List Allocators)

When streaming requires dynamic creation and destruction of individual objects—such as particle buffers, audio stream packets, or entity metadata—pool allocators are used.

  • Fixed-Size Slices: A pool allocator subdivides a single contiguous block into slots of identical size.
  • Zero External Fragmentation: Any freed slot can accommodate any incoming allocation request, eliminating gaps caused by size mismatches.
  • Constant Time (\(O(1)\)) Operations: Unused slots are tracked using an embedded linked list within the free slots themselves, providing deterministic allocation and deallocation times without metadata overhead.

Power-of-Two Allocation (Buddy Allocators)

For variable-sized assets such as varying texture mipmaps or dynamically sized sound buffers, buddy allocators offer a balance between flexibility and fragmentation control:

  • Binary Splitting: The allocator maintains blocks in powers of two. When a request arrives, the smallest suitable power-of-two block is found; if none exist, a larger block is repeatedly halved until it fits.
  • Automatic Coalescing: When an asset is unloaded, the allocator checks if its adjacent "buddy" block is also free. If so, they instantly merge into a single larger block.
  • Mitigating Decay: While internal fragmentation can occur due to rounding to power-of-two boundaries, external fragmentation is drastically reduced because adjacent free blocks continuously recombine.

Virtual Memory Partitioning and Compaction

For large-scale streaming systems, custom allocators interface directly with operating system virtual memory APIs (such as VirtualAlloc on Windows or mmap on POSIX systems):

  • Address Space Reservation: The engine reserves large virtual address ranges upfront without committing physical memory pages. As a streaming sector loads, physical pages are committed; as it unloads, pages are decommitted while retaining the continuous virtual address layout.
  • Handle-Based Indirection (Compaction): Rather than exposing raw memory pointers to game systems, the allocator provides opaque handles or IDs. This allows the allocator to move active assets in physical memory during idle frames to compact free space, updating internal pointer tables transparently without breaking asset references.