Embedding AV1 OBUs in the AVIF mdat Box

The AV1 Image File Format (AVIF) stores compressed image data using the ISO Base Media File Format (ISOBMFF) container architecture. Within an AVIF file, image payloads encoded as AV1 Open Bitstream Units (OBUs) reside inside the mdat (Media Data) box. This article explains how these OBUs are organized within the mdat box, how container metadata references them, and how decoders extract and parse the image stream.

The Role of the mdat Box in AVIF

The mdat box acts as a continuous byte payload container. It holds the raw binary data of the media without defining internal syntax or image properties. For still images, AVIF utilizes the ISOBMFF item mechanism located in the meta box rather than traditional video tracks. The actual coded image data stored in the mdat box consists of one or more AV1 OBUs that decode into a single frame or a sequence of auxiliary frames, such as alpha transparency channels or depth maps.

OBU Structure Inside the Payload

AV1 encodes data as a series of OBUs. Each OBU consists of:

1. OBU Header (1 to 2 bytes): Specifies the OBU type (such as Sequence Header, Frame Header, Tile Group, or Metadata), whether an extension header exists, and whether the obu_has_size_field flag is set.
2. Optional OBU Size Field (LEB128 format): Indicates the length of the following payload in bytes, encoded using a Variable Length Integer (leb128).
3. OBU Payload: The raw compressed data corresponding to the OBU type.

In the mdat box of an AVIF file, the data represents an AV1 bitstream formatted either as a single encapsulated frame or a collection of distinct OBUs. The AVIF specification mandates the "Low Overhead Bitstream Format." Consequently, every OBU stored in the mdat box that does not rely on implicit boundary parsing must have its obu_has_size_field set to 1, ensuring that parsers can delineate between adjacent OBUs within the raw data block.

Locating OBUs via the iloc Box

Because the mdat box does not contain internal indexing for the OBUs, the container's meta box maps the byte positions:

• Item Definition (iinf and infe boxes): Declares an item of type av01 (AV1 Image Item).
• Item Location Box (iloc): Contains the exact byte offset and length specifying where the av01 item starts and ends within the file, pointing directly into the interior of the mdat box.

When an AVIF reader loads the file, it reads the iloc box to determine the byte range within the mdat box that belongs to the primary image item.

Separation Between av1C and mdat

An AV1 decoder requires a Sequence Header OBU to configure parameters such as color space, bit depth, and dimensions before processing coded frames. In AVIF, the Sequence Header OBU can be embedded in two ways:

1. Within the av1C Property: The av1C (AV1 Configuration Box) is stored inside the iprp (Item Properties Box). It typically contains the complete Sequence Header OBU directly in the metadata tree, separate from the mdat box.
2. Within the mdat Payload: The bitstream located inside the mdat box contains the remaining necessary OBUs, usually a Frame Header OBU paired with a Tile Group OBU, or a combined Frame OBU. In some implementations, a duplicate Sequence Header OBU or Temporal Delimiter OBU may also precede the frame data inside the mdat box.

Extraction and Decoding Process

To render the image, the parsing engine executes the following steps:

1. Identifies the primary item ID through the pitm box.
2. Resolves associated properties from the ipma and iprp boxes, obtaining the av1C configuration box containing the AV1 Sequence Header OBU.
3. Queries the iloc box using the primary item ID to locate the offset and byte length of the image payload in the mdat box.
4. Slices the exact byte span from the mdat box containing the serialized OBUs.
5. Feeds the Sequence Header OBU followed by the OBUs extracted from the mdat box into the AV1 video decoder pipeline to reconstruct the uncompressed image pixels.