Hardware-Accelerated AVIF Rendering in iOS and macOS

Apple platforms provide native support for the AV1 Image File Format (AVIF) starting with iOS 16 and macOS 13 Ventura, utilizing dedicated hardware decoding capabilities on Apple Silicon devices equipped with an AV1 decoder (A17 Pro, M3 family, and later). This article details the native frameworks and APIs available across iOS and macOS to achieve hardware-accelerated AVIF decoding and rendering, covering both high-level UI controls and low-level graphics subsystems.

Hardware and System Requirements

To achieve true silicon-level hardware acceleration for AVIF, the host device must feature an Apple chip containing an integrated AV1 hardware decoder:

• iOS: A17 Pro or newer (e.g., iPhone 15 Pro series and later).
• macOS: M3 family (M3, M3 Pro, M3 Max) or newer.

On older chips running iOS 16+ or macOS 13+, the same APIs will still decode AVIF natively, but execution falls back to Apple's optimized software decoding pipeline rather than dedicated fixed-function hardware blocks.

High-Level APIs for AVIF Rendering

1. UIKit (UIImage) and AppKit (NSImage)

The simplest path to rendering AVIF is through the primary image classes in UIKit and AppKit. These classes rely internally on the Image I/O framework and automatically invoke hardware decoding paths when available.

• iOS:

  let image = UIImage(data: avifData)
  // or
  let image = UIImage(named: "sample.avif")

• macOS:

  let image = NSImage(data: avifData)
  // or
  let image = NSImage(contentsOf: fileURL)

2. WebKit (WKWebView)

WebKit provides out-of-the-box hardware-accelerated AVIF rendering for web content and hybrid apps. Standard HTML <img> elements, CSS background images, and <picture> elements referencing AVIF sources are handed off directly to the system's hardware media engine on supported hardware.

<picture>
  <source srcset="image.avif" type="image/avif">
  <img src="image.jpg" alt="Fallback">
</picture>

Low-Level and High-Performance APIs

3. Image I/O (CGImageSource)

The Image I/O framework is the core engine behind image decoding in Apple operating systems. It interfaces directly with low-level decoders to extract CGImage instances from raw image bitstreams.

Key APIs for hardware-accelerated decoding:

• CGImageSourceCreateWithData(_:_:) / CGImageSourceCreateWithURL(_:_:): Initializes the decoder source.
• CGImageSourceCreateImageAtIndex(_:_:_:): Produces a CGImage.

To ensure maximum performance and avoid main-thread latency during decompression:

let options: [CFString: Any] = [
    kCGImageSourceShouldCacheImmediately: true, // Forces decode at call time
    kCGImageSourceShouldAllowFloat: true
]
guard let source = CGImageSourceCreateWithData(avifData as CFData, nil),
      let cgImage = CGImageSourceCreateImageAtIndex(source, 0, options as CFDictionary) else {
    return
}

4. Core Image (CIImage)

Core Image offers GPU-accelerated image processing and filtering. It ingests AVIF natively via Image I/O and creates pipeline-ready CIImage instances, maintaining hardware-decoded representations directly in GPU-accessible memory.

• Initialization: CIImage(contentsOf: url) or CIImage(data: avifData).
• Rendering: Render using a CIContext backed by a Metal device (MTLDevice) to preserve hardware throughput from decode to display.

5. VideoToolbox and Core Video (VTDecompressionSession)

Because AVIF encapsulates an AV1 video frame within an HEIF/ISOBMFF container, developers working with animated AVIF sequences (AVIFS) or custom streaming pipelines can interface directly with VideoToolbox.

• Codec Type: kCMVideoCodecType_AV1
• Session Management: Using VTDecompressionSessionCreate, you can feed AV1 elementary stream packets into the decoder. On devices with A17 Pro or M3-series processors, VideoToolbox routes these frames directly into the hardware AV1 decode pipeline, outputting zero-copy CVPixelBuffer instances (kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange or 10-bit variants) suitable for direct rendering via Metal or Core Animation (AVSampleBufferDisplayLayer).