How Does the Roofline Model Show Compute vs Memory Bounds?

The Roofline performance model is a visual tool used in classical computing to evaluate how efficiently a computational kernel utilizes hardware resources. By plotting attainable performance against arithmetic intensity on a log-log scale, it creates a ceiling shaped like a roof that separates memory-bandwidth-bound workloads from compute-bound workloads.

Core Axes of the Roofline Plot

The Roofline model uses two primary axes, both typically displayed on a logarithmic scale:

  • Horizontal Axis (Arithmetic Intensity): Measured in floating-point operations per byte of memory transferred (\(\text{FLOP/byte}\)). It represents how much computation a kernel performs for each byte moved between main memory and the processor.
  • Vertical Axis (Performance): Measured in floating-point operations per second (\(\text{FLOP/s}\) or \(\text{GFLOP/s}\)). It represents the actual or theoretical computational throughput of the kernel.

Constructing the Roofline Ceilings

Two hardware limits define the upper boundary of the Roofline plot:

  1. Peak Compute Performance (\(\pi\)): The maximum rate at which the processor can execute floating-point operations, measured in \(\text{FLOP/s}\). This forms a flat horizontal line at the top of the chart.
  2. Peak Memory Bandwidth (\(\beta\)): The maximum rate at which data can be transferred between memory and the processor, measured in \(\text{bytes/s}\). On a log-log plot, the memory-bandwidth limit forms a diagonal line with a slope of \(1\), given by:

\[\text{Performance} = \beta \times I\]

where \(I\) is the arithmetic intensity.

Combining these two limits yields the Roofline equation for maximum attainable performance \(P\):

\[P = \min(\pi, \beta \times I)\]

Identifying Compute-Bound vs. Memory-Bandwidth-Bound Kernels

The intersection of the diagonal memory-bandwidth line and the horizontal peak-compute line is called the ridge point. The arithmetic intensity at this intersection is:

\[I_{\text{ridge}} = \frac{\pi}{\beta}\]

This ridge point divides the plot into two distinct regions:

  • Memory-Bandwidth-Bound Region (\(I < I_{\text{ridge}}\)): Kernels plotted to the left of the ridge point fall under the sloped memory-bandwidth roof. Their performance is limited by how fast data can be fetched from memory rather than how fast the processor can compute. Increasing processor speed alone will not improve performance in this region; instead, optimizations must reduce memory traffic or improve data reuse.
  • Compute-Bound Region (\(I > I_{\text{ridge}}\)): Kernels plotted to the right of the ridge point fall under the flat compute roof. Their performance is limited by the processor's peak arithmetic throughput. Memory bandwidth is sufficient to keep the execution units fed, so optimizations should focus on instruction-level parallelism, vectorization, or reducing the total operation count.

Interpreting Kernel Placement

When a kernel's measured performance and arithmetic intensity are plotted as a point on the chart, its vertical distance from the roof indicates optimization headroom. A point sitting directly on the diagonal line is achieving the maximum possible memory bandwidth, while a point sitting on the flat line is saturating the processor's compute units. A point far below the roof reveals inefficiencies such as poor cache utilization, thread divergence, or unoptimized instruction scheduling.