Packaging AV1 Bitstreams in MP4 Containers

Packaging AV1 video bitstreams within the MP4 container—formally known as the ISO Base Media File Format (ISOBMFF)—requires strict adherence to the binding specifications defined by the Alliance for Open Media (AOMedia). This article details the essential structural requirements for encapsulating AV1, including sample entry formats, codec configuration records, Open Bitstream Unit (OBU) alignment, and random-access signaling to ensure seamless playback across media decoders and streaming engines.

The Visual Sample Entry: av01

When carrying AV1 video, the track handler type must be set to vide, and the sample entry in the sample description box (stsd) must use the FourCC code av01. This identifier signals to parsers that the media samples contain AV1 bitstream data.

The av01 sample entry includes standard ISOBMFF visual fields (such as width, height, and display resolution) alongside mandatory AV1-specific descriptors:

• AV1 Configuration Box (av1C): A mandatory child box containing configuration metadata required to initialize the AV1 decoder before parsing elementary samples.
• Color Information Box (colr): An optional box indicating color primaries, transfer characteristics, and matrix coefficients. If present, it should be kept consistent with the color descriptions signaled inside the AV1 sequence headers.

The AV1 Configuration Box (av1C)

The av1C box provides initialization data so the media pipeline knows video parameters without scanning the entire stream. It encapsulates the AV1CodecConfigurationRecord, which contains:

• Marker & Version: A 1-bit marker set to 1 and a 7-bit version field set to 1.
• Profile, Level, and Tier: Fields defining seq_profile, seq_level_idx_0, and seq_tier_0 as specified by the primary Sequence Header.
• Color Characteristics: Bit flags signaling high bit depth (high_bitdepth), twelve-bit mode (twelve_bit), monochrome output (monochrome), and chroma subsampling formats (chroma_subsampling_x, chroma_subsampling_y, and chroma_sample_position).
• Initial Presentation Delay: A flag and integer indicating whether presentation delay is signaled, which assists with buffer management.
• ConfigOBUs: A byte array containing initial Sequence Header OBUs and optionally Metadata OBUs. Carrying the initial sequence header here guarantees the decoder can configure its output pipeline immediately at initialization.

Sample and OBU Formatting Rules

AV1 bitstreams are composed of a sequence of Open Bitstream Units (OBUs). When placed into an ISOBMFF sample, specific framing constraints apply:

• OBU Sizing: Every OBU within an MP4 sample must have the obu_has_size_field bit set to 1. This ensures that each OBU explicitly provides its length, allowing decoders to split units quickly without deep parsing.
• Temporal Delimiter OBUs: Temporal Delimiter OBUs (which signal frame boundaries in elementary transport streams) must be omitted from ISOBMFF samples. The container's sample structures already define temporal boundaries.
• Tile Group and Frame OBUs: An MP4 sample corresponds to a single AV1 temporal unit. It must contain the necessary Frame Header and Tile Group OBUs (or Frame OBUs) required to decode the visual frame for that timestamp.
• Redundant OBUs: Header information (such as Sequence Headers) does not need to be duplicated in every sample if it has already been provided in the av1C box, though repeated sequence headers are permitted before keyframes for mid-stream entry.

Random Access and Synchronization Samples

To facilitate seeking and track switching, MP4 indexing structures must correctly reflect the AV1 bitstream state:

• Sync Samples (Keyframes): A sample is marked as a sync sample (listed in the stss box or marked with sample_is_non_sync_sample = 0 in fragmented MP4) only if it contains an AV1 Key Frame. A sync sample must include an intra-coded frame that can be decoded without reference to past frames, and it should be preceded by a Sequence Header OBU if not already configured.
• Switch Frames: Samples containing AV1 Switch Frames or non-key intra frames can serve as open-GOP or intermediate entry points. These should be identified using sample group descriptions (such as the sap grouping) rather than standard sync sample markers.