AV1 Video in Matroska: MKV Mapping Explained

This article provides an overview of how the AOMedia Video 1 (AV1) bitstream is encapsulated within the Matroska (MKV) multimedia container. Matroska accommodates AV1 by standardizing a specific Codec ID, embedding a configuration record inside the track headers, and packing AV1 Open Bitstream Units (OBUs) into standard Matroska data blocks. Understanding this integration is essential for developers, multimedia engineers, and video enthusiasts looking to package or demux modern AV1 streams efficiently.

The Codec Identifier

In Matroska, every track must define a CodecID string to signal the decoding format to demuxers. For AV1 video tracks, the standard identifier is:

• V_AV1

This string explicitly signals that the elementary stream contained in the track's clusters conforms to the AV1 bitstream specification defined by the Alliance for Open Media (AOMedia).

Track Initialization with CodecPrivate

The CodecPrivate element in the Matroska TrackEntry holds the global configuration data necessary to initialize the AV1 decoder prior to decoding frames.

Matroska adopts the AV1CodecConfigurationRecord format (identical to the structure used in the MP4/ISOBMFF container binding). The CodecPrivate data contains:

• Configuration Version: Specifies the record version (currently version 1).
• Profile and Level: The target AV1 profile (Main, High, or Professional) and operating level.
• Tier and Color Metadata: Flags indicating the tier, bit depth, chroma subsampling format, and color representation.
• Initial Sequence Header OBU: One or more serialized Sequence Header OBUs that define basic sequence parameters such as frame dimensions, color primaries, and temporal configurations.

Embedding the Sequence Header OBU within CodecPrivate guarantees that the decoder can configure its decoding pipeline immediately upon demuxing the track, even before processing the first video frame.

Encapsulating OBUs in Matroska Blocks

AV1 elementary streams consist of a sequence of Open Bitstream Units (OBUs). When mapped into Matroska:

• Temporal Units per Block: Each Matroska Block or SimpleBlock typically encapsulates one complete AV1 Temporal Unit (TU). A Temporal Unit includes all OBUs associated with a single display time, which may consist of one or more frames (such as base frames and overlay/spatial layer frames).
• OBU Framing: OBUs within a block must use the standard OBU header formatting. The length of each OBU is usually indicated by setting the obu_has_size_field flag to 1, providing explicit size boundaries without requiring additional delimiters.
• Redundant OBUs: Temporal Delimiter OBUs, which are commonly found in raw annex-B bitstreams, are generally omitted in Matroska because Matroska's block boundaries already demarcate distinct temporal frames.

Timestamps and Keyframe Detection

Matroska handles presentation timing and seeking through its own track-level metadata:

• Presentation Timestamps (PTS): Each Block has a relative timestamp, which, combined with the segment's TimestampScale, defines when the decoded frame should be presented to the viewer.
• Keyframes: In a Matroska container, a frame is marked as a keyframe (via the keyframe flag in a SimpleBlock or by omitting reference elements in a BlockGroup) if it represents an AV1 Key Frame. In AV1, an intra-coded frame that can be decoded without reference to past frames, coupled with a valid Sequence Header, qualifies as a seekable random-access point.
• Frame Display Management: AV1 features advanced display models, such as using show_existing_frame to display previously decoded reference frames. Matroska handles these by storing the corresponding OBU in a block timed at the exact moment the frame must be shown.