Nested AltRef Frames in AV1 Hierarchical Structures

An AV1 bitstream can fully support multiple nested Alternative Reference (AltRef) frames across hierarchical Group of Pictures (GOP) structures. AV1's advanced reference frame architecture allows encoders to define several non-displayed predictive anchors across different temporal layers within a single mini-GOP. By utilizing nested AltRef frames, AV1 encoders maximize compression efficiency and multi-layer temporal scalability without violating bitstream syntax or memory buffering constraints.

How AltRef Frames Function in AV1

An AltRef frame is a frame that is decoded and stored in the Decoded Picture Buffer (DPB) but not immediately output for display (designated by show_frame = 0). Later in the bitstream, the frame is either used purely as an invisible motion-compensated reference anchor or rendered at its correct presentation time using an overlay frame or the show_existing_frame command.

Nesting AltRef Frames in Hierarchical GOPs

AV1 supports hierarchical, dyadic prediction structures (such as GOP sizes of 16 or 32 frames), where frames are arranged into temporal layers resembling a pyramid:

1. Top-Level AltRef (ARF): The encoder first transmits a future frame spanning the entire mini-GOP as the primary AltRef frame.
2. Intermediate/Nested AltRefs (Sub-ARFs): Between the keyframe (or preceding displayed frame) and the top-level AltRef, the encoder can introduce secondary and tertiary AltRef frames (often referred to as ARF2, ARF3, or internal bi-directional reference frames).
3. Leaf Frames: Lower-level temporal frames then predict bi-directionally between the base reference frames and these intermediate nested AltRefs.

Because each sub-GOP can create its own non-displayed forward reference, these AltRefs are effectively nested within the scope of the higher-level AltRef frames.

Reference Buffer Management

The AV1 specification permits this level of nesting through a flexible reference picture buffer architecture:

• 8 Frame Buffer Slots: The decoder maintains up to 8 reference frame slots in the DPB.
• 7 Reference Frame Types: For predicting any given inter-frame, AV1 provides 7 reference frame pointers: LAST_FRAME, LAST2_FRAME, LAST3_FRAME, GOLDEN_FRAME, BWDREF_FRAME, ALTREF2_FRAME, and ALTREF_FRAME.

By explicitly allocating names such as BWDREF_FRAME, ALTREF2_FRAME, and ALTREF_FRAME, AV1 accommodates multiple forward-looking reference points concurrently. An encoder can refresh specific DPB slots dynamically, allowing deeply nested hierarchical AltRef trees to exist simultaneously as long as the 8-slot physical buffer limit is respected.

Practical Implementation in AV1 Encoders

Production AV1 encoders, including libaom and SVT-AV1, use nested AltRef structures by default in multi-layer hierarchical presets. SVT-AV1, for example, employs a multi-tier hierarchical prediction structure where level-1, level-2, and level-3 frames can act as filtered, non-displayed AltRef anchors. This nesting allows motion compensation to operate over smaller temporal distances, significantly reducing residual errors and improving overall rate-distortion performance.