How Do Hardware Performance Counters Measure CPU Events?

Hardware performance counters are specialized registers built directly into modern processors that record low-level microarchitectural events as code executes. Profilers like Linux perf program these counters through the kernel to measure cache misses, branch mispredictions, and stalled cycles with minimal overhead.

The Performance Monitoring Unit

Modern CPUs include a dedicated hardware block called the Performance Monitoring Unit (PMU). Inside the PMU are two main types of registers:

  • Event select registers: Control registers that tell the hardware which specific microarchitectural event to track.
  • Performance counter registers: Data registers that increment each time the selected event occurs.

Because these counters operate in silicon alongside the execution pipeline, they can observe events that are invisible to software-only timers or instrumentation.

Counting Mode vs. Sampling Mode

Profilers like perf interact with the PMU through the perf_event_open system call, using two primary modes of operation:

  1. Counting mode (perf stat): The kernel resets a counter to zero before a program runs, lets the hardware increment it during execution, and reads the final total when the program finishes. This gives exact or near-exact aggregate counts for events over a run.
  2. Sampling mode (perf record): The profiler configures a counter with a threshold \(N\) (for example, \(N = 100{,}000\) events) by loading the register with \(-N\). When the counter overflows from \(-1\) to \(0\), the PMU raises a hardware interrupt. The kernel's interrupt handler records the instruction pointer, call stack, and process ID, then resets the counter.

Sampling mode allows perf to map hardware bottlenecks back to specific functions and source lines without stopping the CPU on every single event.

Measuring Specific CPU Events

Different microarchitectural structures feed signals into the PMU's event multiplexers:

  • Cache misses: When a load or store instruction misses in the L1, L2, or L3 cache, the cache controller asserts a miss signal. The PMU increments the corresponding cache-miss counter, allowing developers to compute miss rates such as \(\text{Miss Rate} = \frac{\text{Cache Misses}}{\text{Total Cache References}}\).
  • Branch mispredictions: Modern CPUs predict the outcome of conditional branches to keep the pipeline full. When the execution unit resolves a branch and discovers the prediction was wrong, the pipeline flushes the speculative instructions and pulses a misprediction signal to the PMU.
  • Stalled cycles: Out-of-order and superscalar pipelines track cycles where no instructions can be issued or retired due to data dependencies, memory latency, or full execution buffers. The PMU counts these cycles by monitoring pipeline dispatch and retirement stages.

Practical Limitations

While hardware counters are fast and accurate, they have important constraints:

  • Limited registers: A CPU core typically has only a small number of programmable counters (often 4 to 8). If a profiler requests more events than available registers, the kernel multiplexes them over time and scales the results.
  • Interrupt skid: In sampling mode, the CPU may execute several instructions after a counter overflows before the interrupt is serviced, attributing the sample slightly past the exact instruction that caused the event.
  • Speculative execution: Counters may include events from instructions that were executed speculatively and later discarded, unless the event definition explicitly filters for retired instructions.