AV1 Bitstream Recovery Points for Random Access

This article provides an overview of how the AV1 video codec enables random access through specific bitstream recovery points. Random access allows a media player or decoder to seek to an arbitrary position within a video stream and immediately begin valid decoding without parsing the entire preceding stream. In the AV1 specification, this functionality is achieved through three primary mechanisms: Key Frames, Delayed Random Access Points (Delayed RAPs), and Switch Frames (S-Frames), all supported by the Sequence Header Open Bitstream Unit (OBU).

Sequence Header OBUs

For any random access to occur, the decoder must first receive configuration parameters. In AV1, every random access point must be preceded by or bundled with a Sequence Header OBU. This OBU establishes the profile, level, color space, bit depth, and picture dimensions. Without a Sequence Header OBU, a decoder cannot initialize the context necessary to decode subsequent frames, making it an essential prerequisite for all recovery points.

Key Frames

A Key Frame is the most straightforward and traditional recovery point in AV1. When frame_type is set to KEY_FRAME and the frame is displayed (show_frame is set to 1), it indicates a closed Group of Pictures (GOP) boundary.

Key Frames have the following characteristics:

• Intra-Coded: They do not reference any previously decoded frames for inter-frame prediction.
• State Reset: Decoding a Key Frame resets the reference picture buffer and all Cumulative Distribution Function (CDF) probability tables used by the entropy decoder.
• Immediate Playback: Decoders can immediately output the Key Frame and all subsequent frames in presentation order without visual artifacts.

Delayed Random Access Points (Delayed RAPs)

AV1 supports open GOP architectures using Delayed Random Access Points. An open GOP allows frames following an intra-frame in presentation order to reference frames that occurred before the intra-frame in decode order.

In a Delayed RAP:

• The recovery point begins with an intra-coded frame (either a Key Frame that does not immediately reset all state, or an INTRA_ONLY_FRAME).
• Frames decoded after the access point might not be instantly displayable if they rely on references decoded prior to the entry point.
• The bitstream defines a recovery point indicating the exact point (or number of frames) after which the output is guaranteed to be completely correct and free from missing reference errors. This approach improves compression efficiency compared to standard closed GOP Key Frames.

Switch Frames (S-Frames)

Switch Frames (frame_type set to SWITCH_FRAME) are a specialized random access feature in AV1 designed for adaptive bitrate streaming.

An S-Frame allows a decoder to switch from one bitrate track to another without inserting a high-bitrate Key Frame:

• It uses inter-frame prediction, referencing earlier frames decoded at a different quality level or resolution.
• It enforces a complete reset of the entropy coding context (CDFs) to match a predetermined state shared across different stream encodings.
• Because the reference frame buffers are aligned across multiple encodes of the same video content, a player can seamlessly jump tracks at an S-Frame boundary with significantly less overhead than a standard Key Frame.