Build Custom AVIF Decoders with WebCodecs API

The WebCodecs API allows web applications to access low-level media processing components natively implemented in modern browsers. Because the AVIF (AV1 Image File Format) specification fundamentally encapsulates AV1-encoded still images inside an ISOBMFF container, developers can leverage WebCodecs to construct high-performance, custom JavaScript AVIF decoders. This article details the underlying architecture, the implementation workflow, and the benefits and constraints of building custom image pipelines with WebCodecs.

Understanding the AVIF and WebCodecs Relationship

AVIF is derived from the AV1 video standard. Instead of encoding a sequence of moving pictures, an AVIF file stores one or more AV1 keyframes along with metadata like color profiles, alpha channels, and transform properties wrapped inside an ISOBMFF (ISO Base Media File Format) box structure.

The WebCodecs API exposes the browser's internal AV1 decoder via the VideoDecoder interface. Rather than running a heavyweight, software-compiled WebAssembly build of an AV1 decoder (such as dav1d or libaom), JavaScript can pass raw AV1 bitstreams directly to the browser's native, hardware-accelerated decoding engine.

The Custom Decoding Pipeline

A custom AVIF decoder built with WebCodecs relies on a multi-step pipeline:

1. ISOBMFF Demuxing

WebCodecs handles raw video packets, not container formats. The first stage requires a lightweight JavaScript or WebAssembly parser to parse the AVIF file's container boxes (ftyp, meta, hdlr, iloc, and mdat). The parser extracts:

• The raw AV1 Open Bitstream Units (OBUs) containing the primary image payload.
• The auxiliary image item containing alpha channel data, if present.
• Image metadata, such as color matrix coefficients, transfer characteristics, and EXIF/XMP payloads.

2. Decoder Initialization

Once the raw bitstream is isolated, a VideoDecoder instance is created with an output callback and an error handler:

const decoder = new VideoDecoder({
  output: (videoFrame) => {
    // Render or manipulate the decoded frame
    renderFrame(videoFrame);
  },
  error: (err) => console.error("Decoding error:", err)
});

decoder.configure({
  codec: "av01.0.04M.08", // Profile, level, tier, and bit-depth parameters
  hardwareAcceleration: "prefer-hardware"
});

3. Chunk Processing and Decoding

The extracted AV1 data is wrapped in an EncodedVideoChunk and passed into the decoder:

const chunk = new EncodedVideoChunk({
  type: "key",
  timestamp: 0,
  data: av1PayloadUint8Array
});

decoder.decode(chunk);
await decoder.flush();

4. Frame Rendering and Consumption

The output callback receives a VideoFrame object. This frame can be directly drawn to an HTML <canvas> or OffscreenCanvas using CanvasRenderingContext2D.drawImage(), converted into an ImageBitmap via createImageBitmap(), or read as raw pixel data using VideoFrame.copyTo().

Advantages of WebCodecs for AVIF

• Hardware Acceleration: Native hardware decoders process frames with significantly lower CPU usage and battery consumption compared to pure WebAssembly decoders.
• Reduced Bundle Sizes: By offloading AV1 decompression to the browser, JavaScript bundle sizes are reduced from megabytes (for software decoders) to tens of kilobytes (just the ISOBMFF demuxer).
• Controlled Streaming and Progressive Rendering: Applications can stream AVIF files and feed raw chunks incrementally, offering custom loading transitions before standard browser implementations fully decode the image.
• Animated AVIF Flexibility: Decoding multi-frame AVIF files via WebCodecs allows frame-accurate playback control, custom frame-rate scaling, and real-time canvas manipulations.

Technical Considerations and Limitations

• Container Demuxing Overhead: WebCodecs does not parse container formats, placing the responsibility of correctly extracting byte offsets, transformations, and orientation tags on the developer.
• Color Management: AV1 bitstreams contain Color Config OBUs, and AVIF containers can hold ICC color profiles. Rendering the output VideoFrame to a standard Canvas element requires manual alignment with the target color space (such as Display P3 or Rec. 2020) to prevent inaccurate color reproduction.
• Separate Alpha Planes: Unlike formats where transparency is interleaved within pixel data, AVIF often stores alpha channels as a secondary, monochromatic AV1 image track. A fully functional AVIF decoder must run two parallel decoding operations—one for color and one for alpha—and blend them on a Canvas or through a WebGL shader.