How AVIF Handles Auxiliary Images and Depth Maps

The AV1 Image File Format (AVIF) provides native support for auxiliary images, allowing supplementary data like depth maps, alpha channels, and gain maps to be embedded alongside primary image data. By leveraging the ISO Base Media File Format (ISOBMFF) architecture and the AV1 compression codec, AVIF enables efficient storage, separate compression configurations, and standardized referencing for auxiliary data without degrading the quality of the primary image.

The ISOBMFF Architecture and Item References

AVIF relies on ISOBMFF (specifically derived from the High Efficiency Image File Format, or HEIF) to structure its internal items. In AVIF, the primary visible image and any auxiliary images are stored as separate items within the same file.

The container links these items together using the auxl (Auxiliary Image) item reference. The primary image is defined as the reference source, while the auxiliary image (such as a depth map) is linked as the auxiliary target. This separation ensures that standard decoders lacking auxiliary support can simply ignore the auxiliary items and render the primary image without errors.

Auxiliary Type Identification via URNs

To inform decoders how to interpret an auxiliary image, the AVIF specification requires an auxiliary item to specify an Auxiliary Type (AuxiliaryTypeProperty). This property uses a Uniform Resource Name (URN) to define the semantic meaning of the data.

Common auxiliary URN classifications include:

• Alpha/Transparency: Identifies the image as an opacity mask (urn:mpeg:mpegB:cicp:systems:auxiliary:alpha).
• Depth Maps: Identifies geometric distance data from the camera sensor, commonly using standardized or vendor-specific identifiers (such as Apple or Android depth map URNs).
• HDR Gain Maps: Used to store tone-mapping data that adjusts standard dynamic range (SDR) images for high dynamic range (HDR) displays.

Independent AV1 Compression

Unlike formats that pack depth data into unused color channels or append uncompressed raw data to metadata segments, AVIF encodes auxiliary images using dedicated AV1 video codec bitstreams.

• Monochrome Profiles: Depth maps and alpha channels typically represent single-channel scalar values. AVIF utilizes the AV1 monochrome profile (YUV 4:0:0), which omits chroma channels entirely. This drastically reduces file size compared to full-color representations.
• Independent Bit Depths: Depth maps often require higher bit precision (such as 10-bit or 12-bit) to avoid banding artifacts in 3D parallax effects or synthetic depth-of-field calculations. AVIF allows the auxiliary stream to use a higher or lower bit depth than the primary 8-bit or 10-bit color image.
• Custom Quantization: Compression parameters (lossy or lossless) can be tuned specifically for the auxiliary image. A depth map can be compressed aggressively or preserved losslessly depending on the required precision of the application.

Flexible Dimensions and Scaling

AVIF does not require an auxiliary depth map to share the exact pixel dimensions of the primary image. Because depth typically exhibits low spatial frequency compared to high-frequency color textures, AVIF allows the depth map to be stored at a lower resolution (e.g., half-resolution or quarter-resolution). Decoders then upscale the depth map to match the primary image coordinate space during processing, maximizing bandwidth and storage savings.