Perceptual Metrics in AV1: Butteraugli and SSIMULACRA
This article explores how advanced perceptual metrics—specifically Butteraugli and SSIMULACRA—are used to tune AV1 video encoders. While legacy metrics like PSNR and standard SSIM prioritize mathematical pixel differences, perceptual metrics model the biological mechanics of human vision. In modern AV1 development, these metrics guide rate-distortion optimization, adaptive quantization, and psychovisual tuning, enabling encoders to produce visually superior video at significantly lower bitrates.
The Limitation of Traditional Video Metrics
For decades, video encoding relied on Peak Signal-to-Noise Ratio (PSNR) and Mean Squared Error (MSE) to assess compression quality. These metrics calculate the exact numerical difference between original and compressed pixels.
However, the human visual system does not perceive artifacts uniformly. A mathematical error in a high-contrast, complex texture (such as foliage) is virtually invisible to human eyes, whereas the same error in a smooth gradient or a dark scene causes obvious and distracting banding or blocking. When an AV1 encoder optimizes strictly for PSNR, it often blurs textures and wastes bits preserving mathematical fidelity in areas where humans cannot perceive the difference.
What Are Butteraugli and SSIMULACRA?
To overcome the blind spots of legacy metrics, researchers developed psychovisual models that simulate human visual perception:
- Butteraugli: Originally designed by Google for static image evaluation, Butteraugli models the human eye's physical properties. It accounts for spatial frequency sensitivity, color perception (particularly how the human eye is less sensitive to high-frequency changes in blue), and visual masking (the phenomenon where adjacent visual noise obscures artifacts). Butteraugli outputs a score representing the point at which a difference becomes noticeable to the human eye.
- SSIMULACRA (and SSIMULACRA2): Built upon concepts from Butteraugli, structural similarity, and extensive subjective testing databases, SSIMULACRA evaluates perceptual quality with high precision. SSIMULACRA2, the modernized iteration, uses deep psychovisual modeling to score images on a 0–100 scale, capturing compression artifacts like edge smearing, ringing, blockiness, and loss of texture that conventional SSIM overlooks.
How Perceptual Metrics Relate to AV1 Tuning
In modern AV1 encoders, such as mainline SVT-AV1, libaom, and community-driven projects like SVT-AV1-PSY, Butteraugli and SSIMULACRA serve two primary functions: evaluating experimental features and guiding the encoder's internal decision-making processes.
1. Rate-Distortion Optimization (RDO)
Encoders make thousands of decisions per frame regarding block partitioning, prediction modes, and transform sizes. This process, known as Rate-Distortion Optimization (RDO), balances the bitrate cost against the resulting distortion:
\[\text{Cost} = \text{Bitrate} + \lambda \times \text{Distortion}\]
Traditionally, "Distortion" was calculated using simple Sum of Squared Errors (SSE). In psychovisually tuned AV1 implementations, developers modify the distortion calculation to align with principles derived from Butteraugli and SSIMULACRA. Instead of minimizing raw pixel deviations, the encoder minimizes perceptual errors, allowing non-visible mathematical differences if it frees up bits for visible features.
2. Adaptive Quantization (AQ)
Adaptive Quantization determines how aggressively different areas of a frame are compressed. Perceptual metrics highlight the failure points of standard quantization:
- Flat and Dark Areas: Butteraugli identifies high sensitivity to gradients and dark scenes. AV1 tuners use this data to lower quantization in flat regions to eliminate banding.
- Edge Preservation: SSIMULACRA penalizes boundary blurring. Encoders adjust spatial AQ to allocate more bits to strong edges and line art.
- Texture Retention: Rather than smoothing out grain and fine textures to save bits, perceptually tuned AQ retains high-frequency details that prevent the video from looking plastic or unnatural.
3. Automated Encoder Benchmarking and Parameter Search
Developing AV1 presets and tuning flags requires testing thousands of encoding parameters. Subjective human testing is accurate but slow and expensive. SSIMULACRA2 and Butteraugli provide automated, reliable proxies for human eyes.
Developers run automated regressions across large video datasets, scoring variations of AV1 settings against SSIMULACRA2. If a new encoding algorithm improves PSNR but degrades the SSIMULACRA2 score, developers can reject the change knowing it harms real-world viewing quality.
Conclusion
Perceptual metrics like Butteraugli and SSIMULACRA bridge the gap between algorithmic compression and human biology. In AV1 tuning, they redirect encoder priorities away from raw mathematical accuracy and toward visual plausibility. By targeting artifacts that human eyes actually perceive, these metrics allow AV1 to deliver sharper textures, cleaner gradients, and a more faithful visual experience at lower data rates.