Memory Bandwidth Starvation in Multi-Core Benchmarking

This article explains why memory bandwidth starvation frequently limits performance during multi-core hardware benchmarking. It examines the growing disparity between processor core counts and RAM data transfer rates, how simultaneous core utilization overwhelms memory controllers, the role of CPU cache exhaustion, and how limited memory channels ultimately cause processing cores to sit idle waiting for data.

The Disparity Between Core Scaling and RAM Speeds

Over the last decade, CPU manufacturers have rapidly increased the number of physical cores on a single chip. Consumer processors now routinely feature 16 to 24 cores, while enterprise processors can exceed 64 or 128 cores. However, system memory (RAM) bandwidth has not scaled at the same pace.

While transitioning from DDR4 to DDR5 nearly doubled available bandwidth, the number of cores sharing that bandwidth has multiplied much faster. As a result, the amount of memory bandwidth available per core has actually decreased in many modern architectures.

Simultaneous Demand in Benchmarking

In typical daily workloads, only a few CPU cores are active at any given millisecond, leaving plenty of memory bandwidth for the active cores. Multi-core benchmarks, however, are specifically designed to push every available core to 100% utilization at the exact same time.

For a CPU core to process instructions, it must constantly fetch data from the RAM and write results back to it. When 16, 32, or 64 cores demand data simultaneously, the memory controller is flooded with requests. Because the data highway (the memory bus) is only so wide, a traffic jam occurs. This is memory bandwidth starvation: the cores are capable of calculating data faster than the RAM can deliver it.

Cache Exhaustion and Cache Thrashing

Modern CPUs use ultra-fast onboard memory called cache (L1, L2, and L3) to prevent bandwidth starvation. If a core finds the data it needs in the cache, it does not need to access the slower system RAM.

During heavy multi-core benchmarking, two problems occur:

• Dataset Overflow: Many benchmarks use massive datasets that intentionally exceed the size of the CPU's L3 cache. Once the cache is full, cores are forced to fetch data directly from the system RAM.
• Cache Thrashing: Because all cores are working at maximum capacity and often share the same L3 cache pool, they constantly overwrite the cached data of other cores. This reduces cache efficiency and drives up the demand on system memory even further.

The Limitation of Memory Channels

The physical connection between the CPU and the RAM is divided into memory channels. Most mainstream desktop platforms only support dual-channel memory. This means that whether you install an entry-level 6-core CPU or a flagship 24-core CPU on that motherboard, both processors are restricted to the same two data pipes to the RAM.

High-end desktop (HEDT) and server platforms attempt to solve this by offering four, eight, or even twelve memory channels. However, because these platforms also scale up to vastly higher core counts, multi-core benchmarks can still saturate the memory bus.

The Result: Stalled CPU Cycles

When memory bandwidth reaches 100% saturation during a benchmark, the CPU cores experience "wait states" or stalled cycles. The cores remain powered on and allocated to the benchmark task, but they are doing no actual computational work for nanoseconds at a time because they are waiting for the next batch of data to arrive from the RAM. Consequently, the benchmark score stops reflecting the raw compute power of the CPU and instead measures the maximum transfer speed of the system's memory configuration.