Chiplets and Interconnect Speeds in Latency Benchmarking
This article explores how modern chiplet architectures and the speed of their underlying interconnect fabrics influence latency measurements in computer hardware benchmarks. As processor manufacturers move away from single monolithic dies in favor of modular chiplets, the physical distance data must travel increases. Consequently, the speed of the interconnect bridging these chiplets becomes a critical bottleneck. Understanding the relationship between modular silicon and fabric clock speeds is essential for accurately interpreting memory and core-to-core latency benchmarks.
Monolithic vs. Chiplet Architecture
Historically, CPUs were monolithic, meaning all cores, cache, and memory controllers existed on a single piece of silicon. This kept latency low and predictable.
Chiplet architectures break these components into smaller, separate dies connected on a single package. While this improves manufacturing yields and allows for higher core counts, it introduces a physical gap between components. Data can no longer travel exclusively across microscopic internal pathways; it must now be routed out of one chiplet, across a package substrate, and into another. In hardware benchmarking, this extra physical distance inherently increases latency measurements.
Core-to-Core Latency Variations
In a chiplet-based processor, core-to-core latency is no longer uniform. Benchmarks that measure how fast one CPU core can communicate with another will show vastly different results depending on the physical location of the cores being tested.
If two cores reside on the same chiplet, they share a local cache, resulting in ultra-low latency. However, if a benchmark tests communication between cores located on two different chiplets, the data must traverse the interconnect fabric. This cross-chiplet penalty often doubles or triples the latency measurement (measured in nanoseconds) compared to intra-chiplet communication.
Memory Access Latency and the I/O Die
Many chiplet designs place the memory controller on a dedicated Input/Output (I/O) die, completely separate from the compute chiplets. When a CPU core requests data from the system RAM, the request must travel from the compute chiplet, across the interconnect to the I/O die, out to the RAM, and back the same way.
In synthetic memory benchmarks like AIDA64, this routing manifests as noticeably higher memory latency compared to monolithic processors using the exact same RAM speeds and timings. The benchmark is not just measuring the speed of the RAM; it is measuring the round-trip delay of the chiplet interconnect.
The Impact of Interconnect Fabric Speeds
The interconnect fabric—such as AMD's Infinity Fabric or Intel's Ultra Path Interconnect—acts as the highway between chiplets. The clock speed of this fabric directly dictates the severity of the latency penalty.
A faster interconnect fabric reduces the time it takes for data packets to cross from one die to another. During benchmarking, manually overclocking the interconnect fabric yields immediate reductions in both core-to-core and memory latency measurements, even if the CPU core clock speeds and RAM frequencies remain untouched. Conversely, a slow interconnect will bottleneck the fastest RAM and CPU cores, resulting in poor latency benchmark scores.
Synchronization and Clock Ratios
Latency measurements are also heavily influenced by the synchronization between the interconnect fabric and the system memory controller. Interconnects operate with the lowest latency when running at a 1:1 clock ratio with the memory controller and the RAM itself.
If a user installs extremely fast RAM that exceeds the maximum stable frequency of the interconnect fabric, the motherboard will force the fabric into a desynchronized 2:1 mode. Benchmarks immediately penalize this desynchronization. Even though the RAM has higher bandwidth, the latency benchmark will show a sharp spike (worse performance) because data packets must wait for mismatched clock cycles to align before crossing between the memory controller and the interconnect fabric.
Synthetic vs. Real-World Benchmarks
Not all benchmarks react equally to chiplet and interconnect latencies:
- Synthetic Benchmarks: Tools that specifically test memory access times or inter-core communication isolate and expose chiplet delays. Monolithic chips almost always beat chiplet designs in these specific tests.
- Bandwidth-Heavy Workloads: Real-world benchmarks for 3D rendering, video encoding, and file compression rely on raw data throughput and core count. These benchmarks largely ignore latency penalties, allowing chiplet CPUs to dominate due to their higher core counts.
- Latency-Sensitive Workloads: High-framerate gaming and database management require rapid access to small chunks of data. In these benchmarks, interconnect latency directly reduces performance. To mask this latency in benchmarks and real-world use, hardware manufacturers often pack massive amounts of L3 cache directly onto the compute chiplets, reducing the number of times the CPU cores need to use the interconnect fabric to fetch data from the RAM.