AV1 Over CMAF Fragmentation Headers Explained

Streaming AV1 video over the Common Media Application Format (CMAF) requires packaging AV1 bitstreams into standardized ISO Base Media File Format (ISOBMFF) structures. To achieve seamless adaptive bitrate streaming and low-latency delivery, CMAF divides media into an Initialization Segment and a sequence of Media Fragments. This guide outlines the exact fragmentation headers and ISOBMFF boxes required to package and stream AV1 content within CMAF containers.

The CMAF Segmentation Model

CMAF standardizes media delivery by splitting streams into two primary components:

1. CMAF Header (Initialization Segment): Contains track metadata, codec configuration, and initialization parameters, but no media samples.
2. CMAF Fragment (Media Segment / Chunk): Contains a movie fragment header followed by media data, encapsulating AV1 Open Bitstream Units (OBUs).

1. Initialization Segment Headers

The Initialization Segment must contain the following hierarchy of boxes to initialize the AV1 decoder:

• ftyp (File Type Box): Defines the file profile and compatibility. For CMAF AV1, the major brand is typically set to cmfc (CMAF Track), with compatible brands including cmfc, iso8, and av01.
• moov (Movie Box): Encapsulates all presentation metadata.
  • mvhd (Movie Header Box): Declares overall presentation characteristics, such as timescale and duration.
  • trak (Track Box): Defines the individual AV1 video track.
    • tkhd (Track Header Box): Provides visual presentation dimensions (width and height) and track identification.
    • mdia (Media Box): Contains media-specific operational parameters.
      • mdhd (Media Header Box): Specifies track-specific timescale and language.
      • hdlr (Handler Reference Box): Declares the media type, set to vide for video.
      • minf (Media Information Box): Holds handler-specific information.
        • vmhd (Video Media Header Box): Required video-specific configuration.
        • stbl (Sample Table Box): In fragmented media, this box acts primarily as a container for codec configuration.
          • stsd (Sample Description Box): Contains the sample entry for the track.
            • av01 (AV1 Sample Entry): Identifies the video as AV1.
            • av1C (AV1 Configuration Box): Crucial AV1-specific box inside av01. It carries profile details, level, tier, bit depth, color space, and may contain the initial AV1 Sequence Header OBU.
  • mvex (Movie Extends Box): Explicitly indicates that the media is fragmented and that samples will be delivered via subsequent moof boxes.
    • trex (Track Extends Box): Defines default sample values (such as duration, size, and sample flags) for fragments.

2. Media Fragment Headers

Each playable media segment or chunk begins with metadata headers followed by the raw media samples:

• styp (Segment Type Box): Functions similarly to ftyp but is designated for segment files. It typically includes cmfs (CMAF Segment) and msdh (Media Segment) in the compatible brands list.
• moof (Movie Fragment Box): Contains all metadata detailing the media samples located in the immediately following mdat box.
  • mfhd (Movie Fragment Header Box): Carries a monotonically increasing sequence_number to ensure proper packet and fragment ordering.
  • traf (Track Fragment Box): Contains track-level parameters for the current fragment.
    • tfhd (Track Fragment Header Box): References the track_ID defined in the initialization trak box and specifies default flags for the fragment's samples.
    • tfdt (Track Fragment Base Media Decode Time Box): Provides the absolute decode timestamp (baseMediaDecodeTime) for the first sample in the fragment. Using version 1 of tfdt is standard practice to support 64-bit timestamps, preventing overflow during continuous live streaming.
    • trun (Track Run Box): Details the individual sample records. It specifies the number of samples, individual sample durations, sample sizes, presentation time offsets (composition offsets, if B-frames are used), and sample flags (identifying keyframes vs. non-keyframes via sample_is_non_sync_sample).

3. Media Data Payload (mdat)

Directly following the moof box is the mdat (Media Data Box):

• Payload Format: Contains the raw byte stream of AV1 samples described by the trun box.
• OBU Encapsulation: In an ISOBMFF AV1 container, each sample in the mdat represents an AV1 temporal unit consisting of one or more OBUs (such as Sequence Headers, Frame Headers, and Tile Groups). The length-delimited format is applied, meaning each OBU within the sample must include an explicit size field or use length-delimited framing to enable proper parsing by the decoder.