How AVIF Extracts and Stores Film Grain

The AVIF image format preserves cinematic texture efficiently by using AV1's film grain synthesis rather than encoding raw noise as standard image data. Traditional transform coding struggles with film grain, interpreting its high-frequency randomness as complex detail that requires excessive bitrate or gets blurred away. AVIF solves this by denoising the image during encoding, modeling the isolated grain as a set of mathematical parameters, and embedding these instructions into the bitstream so the decoder can synthesize identical-looking grain on demand.

Step 1: Isolating the Grain from the Source

To model film grain, the AV1 encoder must first separate the stochastic noise from the underlying visual content. This separation occurs through an advanced denoising pipeline:

1. Filtering: An edge-preserving smoothing filter (often a temporal filter for video or a bilateral/wavelet spatial filter for still images) processes the source image. This removes high-frequency fluctuations while retaining sharp edges, line art, and structural textures.
2. Residual Calculation: The encoder subtracts the smoothed, denoised image from the original source image. The resulting difference represents the noise residual—the pure grain profile across color channels.
3. Region Partitioning: The residual is analyzed across flat, non-textured regions of the image to ensure that organic subject textures (such as skin pores or fabric weaves) are not falsely identified as film grain.

Step 2: Parameter Extraction and Modeling

Once the residual is isolated, the encoder transforms the pixel-based noise into a compact parametric model. AV1 models film grain using an Autoregressive (AR) process combined with piecewise linear intensity scaling.

• Intensity-Dependent Scaling: Film grain visibility varies with brightness; grain in photographic negative film is typically more pronounced in midtones than in deep shadows or highlights. The encoder computes the standard deviation of the noise at various luminance levels and defines a piecewise linear function. This maps input luminance (Y) and chrominance (Cb, Cr) values to a grain scaling factor using a series of coordinate points (value and scaling weight).
• Autoregressive (AR) Coefficients: Film grain particles exhibit spatial correlation, meaning individual grains clump together in specific patterns. The encoder determines the spatial correlation between adjacent grain pixels by fitting the noise to an AR model. It calculates coefficients that describe how a generated grain pixel depends on its immediate neighboring grain pixels (typically up to a 3x3 or 5x5 lag window).
• Cross-Channel Correlation: The encoder measures whether the grain in the chroma channels correlates with the luma channel, deriving a cross-channel correlation factor to keep colored grain natural.

Step 3: Storing Parameters in the AVIF Stream

AVIF files are structured using the ISO Base Media File Format (ISOBMFF), which wraps underlying AV1 Open Bitstream Units (OBUs). The extracted grain parameters are stored inside the AV1 bitstream metadata rather than the ISOBMFF container boxes.

1. Frame Header OBU Insertion: When writing the AV1 bitstream for the AVIF item, the encoder writes the syntax structure film_grain_params() directly inside the Frame Header OBU.
2. Payload Contents: The film_grain_params() structure requires negligible storage—typically less than 100 bytes—and includes:
   • apply_grain flag: Instructs the decoder to enable the synthesis pipeline.
   • grain_seed: A 16-bit pseudo-random number generator (PRNG) seed that ensures consistent noise generation patterns.
   • scaling_points: Arrays containing the coordinates and values for the piecewise linear intensity functions across Y, Cb, and Cr planes.
   • ar_coeffs: The integer-quantized autoregressive coefficients controlling grain size and clumping behavior.
   • grain_scale_shift and ar_coeff_shift: Bit-shift values used to normalize the fixed-point arithmetic across different hardware decoders.
3. Container Encapsulation: The AV1 Sequence Header and Frame OBUs—now containing the compressed, denoised image data alongside the film_grain_params() metadata—are stored within an mdat (Media Data) box in the AVIF file. The file's metadata maps this item via the hdlr and iinf boxes as the primary image item.

During playback or viewing, an AVIF-compliant decoder decodes the clean, low-bitrate base image, parses the film_grain_params() struct, and executes a deterministic PRNG to synthesize the modeled grain pattern, blending it over the output picture before rendering to the display.