How Does CXL Enable Cache-Coherent Memory Pooling?

Compute Express Link (CXL) is an open industry-standard interconnect built on top of the PCI Express (PCIe) physical layer that enables high-bandwidth, low-latency, cache-coherent communication between CPUs and accelerators such as GPUs, FPGAs, and smart NICs. By unifying memory spaces and maintaining hardware-enforced cache coherency, CXL allows multiple processors and accelerators to share and pool memory resources dynamically without relying on slow software-managed data copies.

The Foundation: CXL Protocols Over PCIe

CXL operates over the PCIe electrical and physical infrastructure, using PCIe 5.0 and 6.0 signaling rates while replacing the traditional transaction layer with three specialized protocols multiplexed onto a single link:

  • CXL.io: Handles device discovery, configuration, initialization, register access, and traditional non-coherent DMA transfers, functioning similarly to standard PCIe.
  • CXL.cache: Allows attached accelerators to cache host CPU memory locally. The host manages coherency, ensuring that if the CPU or another device modifies a cached address, the accelerator's local copy is invalidated or updated automatically.
  • CXL.mem: Enables the host CPU to access memory attached to a CXL device (such as an accelerator's HBM or a dedicated memory expander) using standard load and store instructions, rather than explicit DMA commands.

Device Types and Coherency Models

CXL defines three primary device types based on which protocols they implement:

  • Type 1 Devices: Accelerators without local memory (such as smart NICs) that use CXL.io and CXL.cache to coherently cache host memory.
  • Type 2 Devices: Accelerators with their own high-bandwidth local memory (such as GPUs and FPGAs) that use all three protocols (CXL.io, CXL.cache, and CXL.mem). This allows the CPU to coherently access the accelerator's memory and the accelerator to coherently access the CPU's memory.
  • Type 3 Devices: Memory expansion buffers and pooled memory appliances that use CXL.io and CXL.mem to provide additional capacity and bandwidth to hosts.

In a traditional PCIe setup, sharing data between a CPU and a GPU requires explicit memory copies across the bus, software synchronization, and duplicate data structures. With CXL Type 2 devices, both the CPU and the accelerator operate on a single shared virtual and physical address space. Hardware snoop filters and directory-based coherency protocols track cache-line states (Modified, Exclusive, Shared, Invalid) across the link, eliminating redundant copies and reducing synchronization overhead.

From Direct Attachment to Memory Pooling

While CXL 1.0 and 1.1 focused on point-to-point connections between a single host and a single device, CXL 2.0 introduced CXL switching and Single Logical Device (SLD) / Multi-Logical Device (MLD) architectures for memory pooling.

Through a CXL switch, a rack or server can house a pool of Type 3 memory devices or Type 2 accelerators that are not permanently bound to a single CPU. The fabric manager dynamically allocates slices of memory from the pool to different host CPUs based on workload demand. For example, if Host A requires an extra 256 GB of memory for an in-memory database and Host B is idle, the CXL switch can bind those memory regions to Host A's physical address space. When Host A finishes, the memory can be zeroed and reassigned to another host.

CXL 3.0 and 3.1 further extend this capability by introducing fabric-attached memory and peer-to-peer (P2P) coherency. Multiple hosts and accelerators can now access the same shared memory region simultaneously with hardware-enforced coherency, enabled by back-invalidation channels in CXL.mem. This allows accelerators to exchange intermediate results directly through pooled memory without routing data through a host CPU.

Performance and Architectural Impact

By decoupling memory capacity and bandwidth from individual CPU sockets, CXL addresses the "memory wall" in classical computing. Instead of over-provisioning every server with maximum DRAM to handle peak workloads, data centers can provision average memory per node and draw from a shared CXL pool as needed.

If a system has \(N\) hosts each requiring peak memory \(M_{\text{peak}}\) but average memory \(M_{\text{avg}}\), traditional provisioning requires \(N \times M_{\text{peak}}\) total memory. With CXL pooling, total provisioned memory approaches \(N \times M_{\text{avg}} + M_{\text{burst}}\), significantly reducing stranded memory, power consumption, and hardware cost while maintaining cache-coherent access across CPUs and accelerators.