Structure of Arrays vs Array of Structures in C++

In C++, choosing between the Array of Structures (AoS) and Structure of Arrays (SoA) memory layouts fundamentally dictates how efficiently hardware executes code. This article examines the architectural differences between AoS and SoA, detailing how CPU cache line utilization, SIMD vectorization, and memory access patterns determine runtime performance, along with practical guidelines for selecting the optimal layout for performance-critical systems.

Understanding AoS and SoA

The Array of Structures (AoS) pattern represents the classic object-oriented paradigm. Individual entities are modeled as C++ struct or class types, and instances are placed contiguously in memory within a container such as std::vector:

struct Particle {
    float x, y, z;
    float vx, vy, vz;
    float mass;
};

std::vector<Particle> particles;

In AoS, all attributes for a single entity reside sequentially in memory.

Conversely, the Structure of Arrays (SoA) follows a data-oriented design. Instead of wrapping individual components into a composite object, each attribute forms its own dedicated array:

struct Particles {
    std::vector<float> x, y, z;
    std::vector<float> vx, vy, vz;
    std::vector<float> mass;
};

Particles particles;

Here, identical components across all entities are grouped contiguously in memory.

Cache Line Utilization and Spatial Locality

Modern CPUs do not fetch individual bytes from main memory; they load fixed-size blocks—typically 64 bytes—known as cache lines. Performance largely depends on whether an application uses all the data transferred into the cache.

  • AoS under partial updates: If a physics loop only updates positions using velocities (x += vx), it reads fields x, y, z, vx, vy, and vz, while ignoring mass. Despite being unused, mass still occupies space within each loaded 64-byte cache line. This introduces cache pollution and wastes memory bandwidth.
  • SoA under partial updates: When processing only the x component, the CPU loads a cache line filled exclusively with consecutive x coordinates. Sixteen single-precision floats (64 bytes / 4 bytes) fit into a single cache line without any padding or irrelevant fields. Memory bandwidth efficiency reaches near 100%, and L1/L2 data cache hit rates increase dramatically.

However, if an algorithm processes all attributes of a single entity at once (such as initializing, serializing, or transforming a specific particle), AoS exhibits superior spatial locality, whereas SoA would require the CPU to maintain multiple independent cache lines simultaneously.

SIMD and Auto-Vectorization

Single Instruction, Multiple Data (SIMD) instruction sets (such as SSE, AVX-256, and AVX-512) allow the CPU to perform operations on multiple data points in a single clock cycle.

  • Vectorization in SoA: Because homogenous data is contiguous, compilers can trivially map SoA arrays directly to vector registers. For example, using AVX2, eight consecutive x floats can be loaded with a single _mm256_load_ps instruction. Mathematical operations run at maximum throughput without rearrangement.
  • Vectorization in AoS: Because identical fields are interleaved with other member variables, vectorized operations require strided loads. The CPU must either rely on costly gather/scatter instructions or spend cycles shuffling and unpacking registers before and after performing computations. Consequently, compilers frequently fail to auto-vectorize loops written over AoS layouts.

Structural Trade-offs

While SoA improves throughput for high-volume, batch-processed workloads, it introduces engineering trade-offs:

  1. API Ergonomics: Passing an individual entity across function boundaries is trivial in AoS (void update(Particle& p)), but cumbersome in SoA, often requiring an index-based reference or custom proxy object.
  2. Dynamic Operations: Inserting, deleting, or resizing elements in SoA requires synchronizing operations across multiple distinct dynamic arrays.

The Hybrid Approach: AoSoA

To reconcile the benefits of both patterns, high-performance C++ applications often employ an Array of Structures of Arrays (AoSoA), also known as tiled SoA:

struct ParticlePacket {
    float x[8], y[8], z[8];
    float vx[8], vy[8], vz[8];
};

std::vector<ParticlePacket> particles;

By sizing the inner fixed arrays to match the width of the target SIMD register (e.g., 8 floats for 256-bit AVX), AoSoA enables direct aligned SIMD loads while preserving the modular allocation, sorting, and entity-management benefits of AoS.