AVIF Audio and Text Tracks for Accessible Diagrams

AVIF offers superior compression for still and animated graphics, but its handling of embedded audio and text tracks for animated accessible diagrams is strictly limited by the format’s specification and browser implementation. While the underlying container format technically supports multimedia multiplexing, AVIF decoders in web browsers and image viewers intentionally restrict processing to visual data. Consequently, making animated diagrams accessible through AVIF requires understanding container boundaries and relying on web-layer accessibility mechanisms rather than embedded media tracks.

Container Architecture vs. AVIF Specification

The AV1 Image File Format (AVIF) is built upon the ISO Base Media File Format (ISOBMFF), the same standardized container structure used by MP4. In theory, ISOBMFF can encapsulate synchronized video, audio, and timed text tracks (such as WebVTT or TTML).

However, the AVIF specification explicitly defines profiles for still images and animated image sequences (using the avis brand). Because the format is strictly defined as an image format rather than a general-purpose audiovisual container, compliance guidelines focus exclusively on visual frames encoded with the AV1 video codec.

Embedded Audio Handling

AVIF does not support embedded audio tracks. If an audio track (such as AAC or Opus) is multiplexed into an ISOBMFF file alongside an AV1 image sequence, the file no longer conforms to standard AVIF profile constraints.

When a web browser encounters an .avif file with an embedded audio track loaded via an <img> tag or CSS background, one of two things happens:

• Playback Failure: The decoder rejects the file because the presence of non-conforming tracks violates the image MIME type (image/avif).
• Track Stripping: The image parser ignores all non-visual tracks completely, rendering only the silent animation frames.

For animated diagrams requiring spoken explanations, sound effects, or auditory cues, AVIF cannot act as a self-contained multimedia carrier.

Embedded Text Tracks and Metadata

Accessibility for diagrams often relies on timed text tracks, such as captions, screen-reader descriptions, or contextual callouts that align with specific steps in an animation.

• Timed Text Tracks: While ISOBMFF supports timed text tracks, standard AVIF decoders do not expose or parse these tracks to accessibility APIs or the Document Object Model (DOM). Embedding a timed text track directly within an AVIF file will not generate captions or provide screen-reader events in browsers.
• Static Metadata: AVIF natively supports static metadata payloads, including Exif and XMP. Authors can embed descriptive text, author details, and licensing information within these metadata boxes. However, this metadata is static; it cannot synchronize dynamically with the frames of an animated sequence, and browsers do not automatically surface XMP descriptions to assistive technologies.

Best Practices for Accessible Animated Diagrams

Because AVIF cannot natively deliver synchronized audio or interactive text tracks, developers must handle accessibility at the application and markup layers:

1. Use HTML5 Video for True Multimedia: If an animated diagram requires embedded audio and synchronized captions, use the HTML5 <video> element with an AV1-encoded MP4 or WebM container instead of AVIF. This approach natively supports <track> elements for WebVTT captions, audio descriptions, and multiple audio tracks.
2. Accessible DOM Fallbacks: When deploying animated AVIF via the <img> tag, provide programmatic text alternatives using HTML attributes such as alt, aria-describedby, or an adjacent <figcaption> containing a detailed step-by-step description of the diagram's phases.
3. SVG Integration: For complex, vector-based diagrams, interactive SVG is often preferable to raster AVIF. SVG allows individual diagram elements to be annotated with ARIA attributes, text labels, and synchronized visual cues readable by screen readers.