How AV1 Dynamic Reference Frame Selection Works

This article provides an overview of the dynamic reference frame selection mechanism in the AV1 video codec. AV1 significantly improves compression efficiency over previous codecs by expanding the pool of available reference frames and introducing flexible, frame-level and block-level mechanisms to select them. The following sections explain the architecture of AV1's frame buffers, how the encoder evaluates reference candidates, how motion vectors adapt dynamically, and how reference updates are signaled.

The 8-Slot Buffer and 7-Reference Structure

Unlike older codecs like VP9 (which allowed up to three reference frames) or standard configurations of H.264, AV1 maintains a Decoded Picture Buffer (DPB) storing up to eight candidate frames. For predicting any given inter-frame, AV1 can select up to seven different reference frames from this pool:

• Past references: LAST, LAST2, LAST3, and GOLDEN
• Future/Alternative references: BWDREF (Backward Reference), ALTREF2 (Alternative Reference 2), and ALTREF (Alternative Reference)

The terms "past" and "future" refer to display order. In coding order, future frames are encoded in advance (using hierarchical prediction structures) and placed in the reference buffer to serve as backward reference targets for bidirectional prediction.

Frame-Level Buffer Assignment

Before individual blocks are encoded, AV1 dynamically updates its virtual reference frame slots at the frame header level. An encoder decides:

1. Which slots to assign to the active references: The encoder maps the seven logical reference names (LAST through ALTREF) to specific physical slots in the 8-frame buffer.
2. Which slots to refresh: After a frame is decoded, the encoder uses a bitmask (refresh_frame_flags) to dynamically determine which buffer slot will be overwritten by the newly reconstructed frame.

This dynamic reassignment allows the encoder to keep key background frames (such as GOLDEN or ALTREF) in memory across long temporal distances without re-encoding them.

Block-Level Candidate Evaluation and RDO

At the coding unit (block) level, the encoder uses Rate-Distortion Optimization (RDO) to determine the best reference frame.

Because testing all seven references for every partition size would cause high computational complexity, AV1 encoders use pruning strategies during the dynamic selection process:

• Early Skip Detection: If a reference frame produces a negligible residual error with minimal motion vectors, further reference searches are skipped.
• Spatial Correlation: Neighboring blocks provide an initial list of most-likely references.
• Motion Search Hierarchy: A fast motion estimation pass tests prospective references; only references showing low distortion continue to full sub-pixel and transform evaluation.

Single and Compound Prediction Modes

AV1 dynamically allows blocks to use either single reference prediction or compound reference prediction (combining two references simultaneously).

In compound mode, references are grouped dynamically into valid pairs:

• Bidirectional Compound: Pairs one past reference (LAST–GOLDEN) with one future reference (BWDREF–ALTREF).
• Unidirectional Compound: Pairs two past frames or two future frames to account for sudden lighting changes, zooms, or partial occlusions.

AV1 computes a weighted average or uses mask-based blending (wedge prediction) between the two chosen reference frames, dynamically optimizing how information from each reference is blended into the final predictor.

Motion Field Motion Vector (MFMV) Projection

To improve the accuracy of dynamic reference selection, AV1 uses Motion Field Motion Vector (MFMV) projection. The codec projects motion vectors from previously decoded frames onto the current frame's temporal plane based on the exact temporal distance (display order distance) between the current frame and its references.

This temporal scaling ensures that even if a block selects a reference frame that is temporally distant, the initial motion vector predictor is accurately scaled to match the speed and direction of moving objects.

Entropy-Coded Signaling

Dynamic selections must be transmitted efficiently. AV1 minimizes the bit overhead of referencing up to seven frames by using context-adaptive binary arithmetic coding (CABAC).

The reference frame syntax uses a binary tree signaling structure conditioned on the references selected by spatial neighbors (left and above blocks). If a block's neighbors frequently select LAST or BWDREF, the probability model assigns fewer bits to those indices, dynamically adapting the bitstream overhead to local video content.