HLS AV1 Video Fragmentation for Apple Delivery

This article provides an overview of how HTTP Live Streaming (HLS) delivers AV1-encoded video to Apple devices using fragmented MP4 (fMP4) containers. It covers the structural requirements of Common Media Application Format (CMAF) media segments, initialization box mapping, codec-specific playlist tagging, and how Apple’s native media framework (AVFoundation) parses and renders hardware-accelerated AV1 streams across compatible hardware like the A17 Pro and M3 chip families.

The Shift to Fragmented MP4 (fMP4)

HLS traditionally utilized MPEG-2 Transport Streams (TS) for legacy codecs like H.264. However, Apple’s HLS specification mandates the use of fragmented MP4 (ISO/IEC 14496-12) or CMAF (Common Media Application Format) containers for modern codecs, including AV1.

AV1 video streams are encapsulated using the av01 four-character code (FourCC) inside the ISO Base Media File Format (ISOBMFF). Legacy .ts containers do not support AV1; delivery requires splitting the video stream into an initialization segment and a series of discrete media fragments.

Structure of AV1 Segments in HLS

HLS delivers fragmented AV1 by separating metadata from compressed sample data:

  1. Initialization Segment: Contains the ftyp (File Type) and moov (Movie) boxes. This file defines the track header, color profile, and the av1C (AV1 Configuration Box) containing the Sequence Header Open Bitstream Unit (OBU). The initialization segment is downloaded once per stream variation.
  2. Media Segments: Composed of paired moof (Movie Fragment) and mdat (Media Data) boxes.
    • The moof box contains the fragment metadata, including sample run parameters (trun) and decode timestamps.
    • The mdat box carries the actual AV1 temporal units (OBUs containing frame headers and tile groups).

For reliable playback in the Apple ecosystem, each fragment must begin with a closed Group of Pictures (GOP) starting on an AV1 Key Frame (IDR equivalent) to ensure independent decodability.

HLS Multivariant and Media Playlist Implementation

To properly signal AV1 content to Apple’s native player framework (AVPlayer), the HLS master and media playlists must include specific tags and RFC 6381-compliant codec parameters.

Multivariant Playlist (master.m3u8)

The multivariant playlist must declare the full AV1 codec string within the #EXT-X-STREAM-INF tag, specifying profile, level, and bit depth:

#EXTM3U
#EXT-X-VERSION:7

#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1920x1080,CODECS="av01.0.08M.10"
1080p/prog_index.m3u8

Media Playlist (prog_index.m3u8)

The media playlist maps the initialization segment and indexes the media fragments sequentially:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD

#EXT-X-MAP:URI="init.mp4"

#EXTINF:6.00000,
segment_0.m4s
#EXTINF:6.00000,
segment_1.m4s
#EXT-X-ENDLIST

The #EXT-X-MAP tag points directly to the init.mp4 file, ensuring AVPlayer configures the decoder pipeline before attempting to decode subsequent .m4s fragments.

Apple Platform Compatibility and Hardware Decoding

Handling AV1 via HLS depends heavily on platform hardware. AV1 hardware acceleration was introduced with the A17 Pro (iPhone 15 Pro and newer) and the M3 series chips (Mac and iPad).

On supported devices:

Low-Latency HLS (LL-HLS) with AV1

AV1 fragmentation also integrates with Low-Latency HLS. In LL-HLS configurations, media segments are broken down into smaller sub-segments called "parts" (using fragmented MP4 chunks). These parts are served using HTTP/2 or HTTP/3 chunked transfer encoding, allowing Apple client devices to begin decoding parts of an AV1 GOP before the full segment has finished generating on the packager.