How AV1 Decoders Construct Reference Buffers

In the AV1 video coding standard, the decoder maintains an internal Decoded Picture Buffer (DPB) consisting of eight virtual frame slots to manage advanced temporal prediction and reference sharing. This article explains how an AV1 decoder processes reference buffer reconstruction when encountering an unannounced or non-displayed frame, detailing the signaling mechanisms, buffer slot mapping, and memory update procedures required to keep the decoding pipeline synchronized without immediately outputting pictures to the display queue.

The Nature of Unannounced Frames in AV1

In AV1, an "unannounced" or invisible frame is typically a frame coded with the header syntax element show_frame set to 0. These frames are decoded and stored directly into the reference picture buffer rather than being sent to the display pipeline. They often serve as forward-looking prediction sources (such as Alternate Reference frames, or ALTREF), intermediate overlay surfaces, or non-displayed intra frames intended to prime reference buffers during random access or scene transitions.

Because these frames do not correspond to an immediate display event, the decoder must construct and update its reference buffer state solely based on the structural header instructions accompanying the frame.

The 8-Slot Reference Architecture

The AV1 decoder maintains an internal pool of eight reference frame slots (ref_frame[8]). Active inter-prediction tools access these slots through seven reference frame aliases:

• LAST_FRAME, LAST2_FRAME, LAST3_FRAME: Past temporal references.
• GOLDEN_FRAME: A designated past reference often held across longer intervals.
• BWDREF_FRAME, ALTREF2_FRAME, ALTREF_FRAME: Backward or lookahead temporal references.

When an unannounced frame enters the decoder, it may draw from any of these slots for its own inter-prediction, or it may be coded as an intra-only frame that requires no prior references.

Reference Construction and Slot Mapping Mechanism

Upon receiving an unannounced frame header, the decoder executes a precise sequence to initialize and update its buffer state:

1. Frame Type and Reference Header Evaluation

The decoder inspects frame_type and primary_ref_frame:

• If the frame is an intra-only frame (KEY_FRAME or INTRA_ONLY_FRAME), the decoder does not map motion compensation references. If it is a Key Frame, it typically invalidates or resets all reference slots.
• If primary_ref_frame is specified, the decoder copies foundational parameters (such as segmentation maps, loop filter configurations, and global motion models) from that designated buffer index. If it is set to PRIMARY_REF_NONE, these structures are reinitialized to defaults.

2. Association of Reference Pointers (ref_frame_idx)

If the unannounced frame is an inter frame, the bitstream supplies the ref_frame_idx array. This array explicitly directs the decoder on which of the eight physical buffer slots correspond to each of the prediction references (LAST_FRAME through ALTREF_FRAME). The decoder loads the pixel data and motion vectors associated with those slots into the prediction pipeline.

3. Image Reconstruction

The decoder reconstructs the unannounced frame by parsing transform blocks, generating intra- or inter-predictions from the assigned reference slots, applying inverse transforms, and adding residual signals. Loop restoration, CDEF (Constrained Directional Enhancement Filter), and deblocking filters are applied according to the frame header flags.

4. Buffer Refresh Signaling (refresh_frame_flags)

The defining step for storing an unannounced frame is evaluating the 8-bit refresh_frame_flags syntax element. Each bit in this mask corresponds to one of the eight reference slots:

• A bit value of 1 instructs the decoder to overwrite that buffer slot with the newly decoded, reconstructed frame.
• A bit value of 0 retains the existing reference picture in that slot.

By setting show_frame = 0 alongside specific refresh_frame_flags, an unannounced frame can overwrite one or more slots (such as replacing an outdated ALTREF) without ever being presented to the display device.

Handling Unannounced Frames with show_existing_frame

A variant of the unannounced frame mechanism occurs when a bitstream signals show_existing_frame = 1. In this scenario, no new pixel decoding takes place. Instead:

• The header specifies an index (frame_to_show_map_idx) pointing to an existing reference slot populated earlier by a non-displayed frame.
• The decoder reads the frame directly from that reference buffer and outputs it to the display.
• If the frame is flagged as a key-frame replacement via reset_decoder_state, the decoder updates reference metadata accordingly to re-anchor subsequent predictions.

Recovery from Missing Reference Buffers

If an unannounced frame arrives following packet loss or an out-of-order seek, the target reference slots referenced by ref_frame_idx might be invalid or uninitialized. Under strict specification compliance, decoders flag these frames as unreferenceable. Robust decoders employ concealment techniques by either synthesizing reference memory with default neutral-gray values (luma 128, chroma 128) or replicating the nearest valid reference picture in the buffer to prevent memory faults while reconstructing the frame.