Google Benchmark: Measure Latency and Cache Misses in C++

This article explains how Google Benchmark enables C++ developers to evaluate micro-level performance by accurately measuring nanosecond execution latency and CPU cache misses. It covers the framework's internal timing mechanisms, anti-optimization techniques, and its integration with system hardware counters via performance subsystem APIs to capture cache events alongside raw execution times.

Measuring Nanosecond Latency

Google Benchmark measures sub-microsecond and nanosecond execution times by repeatedly invoking a target code block inside a timed iteration loop (for (auto _ : state)). Because a single execution of a micro-operation often falls within the margin of error of operating system timers, the framework automatically scales the iteration count until the cumulative runtime exceeds a configurable threshold (typically around one second). The total elapsed time is then divided by the total number of iterations to yield the average duration per operation.

To capture these tiny intervals with nanosecond resolution, Google Benchmark relies on OS-specific high-resolution monotonic clocks, such as clock_gettime(CLOCK_MONOTONIC) on Linux or QueryPerformanceCounter on Windows. When supported, it can also leverage hardware cycle counters like the x86 RDTSC instruction.

Accurate nanosecond benchmarking also requires preventing the compiler from optimizing away operations whose outputs are not used. Google Benchmark addresses this with two essential primitives:

  • benchmark::DoNotOptimize(value): Forces the compiler to treat the expression's result as read or written through an inline assembly block with memory clobbers, preventing dead-code elimination.
  • benchmark::ClobberMemory(): Forces the compiler to flush pending writes to memory and reload data, preventing optimizations that hoist memory access entirely out of the timing loop.

Measuring Cache Misses

While runtime is captured via monotonic software clocks, measuring CPU cache behavior requires reading Hardware Performance Counters (HPCs). These are physical registers inside modern processors (such as Intel, AMD, and ARM) that increment whenever specific microarchitectural events occur, including L1 data cache misses, instruction cache misses, and Last-Level Cache (LLC) misses.

Google Benchmark interfaces with these counters primarily through the Linux perf_event_open system call. The measurement workflow operates as follows:

  1. Configuration: Benchmarks define performance counter metrics via command-line flags (such as --benchmark_perf_counters=CYCLES,INSTRUCTIONS,CACHE-MISSES,L1-DCACHE-LOAD-MISSES) or programmatically using custom counter hooks.
  2. Sampling: At the start of the benchmark run, the framework opens file descriptors targeting the specified hardware events for the calling thread.
  3. Delta Calculation: The counters are read immediately before and after the timing loop runs. The framework subtracts the initial counter value from the final value, normalizes the total against the number of iterations, and reports the event frequency (e.g., cache misses per iteration or miss rates relative to total instructions).

By combining high-resolution monotonic timers with direct access to CPU performance monitoring units, Google Benchmark isolates the mechanical costs of code execution down to individual instructions and memory access stalls.