How AV1 Alternate Reference Frames Improve Compression

In modern video encoding, the AV1 codec achieves significant bandwidth efficiency by utilizing Alternate Reference (altref) frames—internally generated frames that are decoded into reference buffers but never directly rendered on the screen. By acting as invisible, synthetic anchors generated via temporal filtering across multiple source frames, altrefs provide surrounding display frames with an exceptionally clean, noise-free prediction target. This mechanism drastically cuts down the residual data that visible frames must encode, yielding higher visual quality at lower bitrates.

The Concept of Invisible Frames

Standard inter-frame prediction in video codecs typically references previously displayed frames to predict the content of future frames. AV1 expands this paradigm with non-display frames. When an encoder determines that a sequence contains repetitive motion, panning, or consistent background elements, it can insert an altref frame.

The decoder receives the compressed altref frame, reconstructs it, and places it into one of its reference frame slots. However, the decoder's display pipeline bypasses outputting this frame to the monitor. Instead, subsequent visible frames use this stored frame as a baseline for motion vectors and texture data.

Motion-Compensated Temporal Filtering (MCTF)

The primary reason an altref frame enhances compression is how it is synthesized. Encoders construct altref frames by looking ahead across a window of source pictures—typically spanning several frames into the future—and applying Motion-Compensated Temporal Filtering (MCTF).

Camera sensors inevitably introduce high-frequency temporal noise (grain or sensor jitter). If an encoder attempts to preserve this noise across individual frames, it wastes a large percentage of its bitrate budget. During the MCTF process, the encoder:

1. Tracks motion trajectories of blocks across the lookahead window.
2. Aligns corresponding pixels along these motion vectors.
3. Blends these pixels using a weighted average.

This temporal averaging removes random sensor noise while preserving authentic scene edges and textures. The resulting altref frame is a super-sampled, "denoised" representation of the scene that represents the underlying visual structure better than any single raw camera frame could.

Minimizing Residual Error in Visible Frames

Compression efficiency relies on minimizing the "residual"—the difference between the motion-predicted image and the actual image.

Because the altref frame is heavily denoised and structurally stable, visible frames that predict from it generate much smaller residual signals. When a visible frame points its motion vectors to the altref frame:

• Static and slow-moving areas require virtually zero correction data, allowing the encoder to assign near-zero bits to those blocks.
• Complex textures remain sharp because the reference target is free of erratic, frame-to-frame noise fluctuations.
• Bit allocations can be shifted from mundane background maintenance to fast-moving or newly revealed scene elements.

Forward and Bidirectional Prediction Advantages

AV1 allows individual blocks within a frame to reference up to two sources simultaneously (compound prediction) chosen from up to eight available reference slots. Altref frames are often placed temporally ahead of the visible frames that reference them.

This forward-looking placement allows visible frames to be bidirectionally interpolated between a past visible frame and a future altref frame. Occlusions, lighting changes, and objects entering or leaving the camera view are significantly easier to resolve when the encoder can sample from a clean snapshot of what the scene will look like moments later.

Frame Reuse and the "Show Existing Frame" Tool

Once the group of pictures referencing the altref frame has passed, the encoder can bring the altref frame to the screen using the show_existing_frame syntax. If a future display frame closely matches the already-decoded altref frame, the bitstream simply issues a command to display the stored buffer. This effectively yields a full frame of high-quality video for a fraction of the data overhead required to encode an independent frame from scratch.