How Recursive Filtering Works in AV1 Intra Modes

This article provides an overview of how recursive filtering operates within the intra prediction framework of the AV1 video codec. It examines the mechanics of dividing blocks into smaller sub-units, applying pre-trained filter matrices, and using newly predicted samples to iteratively derive adjacent predictions. Readers will gain a clear technical understanding of how this approach improves compression efficiency compared to conventional directional intra prediction, along with the performance trade-offs involved.

The Limits of Standard Intra Prediction

Standard intra prediction in modern video codecs typically relies on directional extrapolation. Boundary samples from previously reconstructed neighboring blocks (above and to the left) are projected straight across the target block along a specified angle. While efficient for simple patterns, this approach struggles over longer distances within large blocks. As the distance from the reference boundary increases, prediction accuracy decreases, often leading to smooth or blurry textures that require high residual bitrates to correct.

The Core Concept of Recursive Filtering

Recursive filtering—most prominently realized in AV1 as Filter Intra Mode—addresses the degradation of reference samples by treating intra prediction as an iterative, autoregressive process. Instead of projecting outer boundary samples across the entire block in a single step, recursive intra prediction subdivides the block into smaller patches (typically 4x2 or 4x4 sub-units) and filters them sequentially.

The primary innovation of this approach is feedback: once a sub-unit is predicted, its newly synthesized pixel values immediately serve as the reference samples for the neighboring sub-units directly to its right and below.

Step-by-Step Mechanics of the Recursive Process

The recursive prediction workflow operates through the following discrete steps:

  1. Sub-Block Partitioning: The target block (such as an 8x8 or 16x16 transform block) is split into a grid of smaller sub-blocks, commonly sized at 4x2 pixels.
  2. Initial Boundary Sampling: For the top-left sub-block, the algorithm extracts true reconstructed samples from the adjacent external boundaries (the top and left neighbor blocks).
  3. Matrix-Based Filtering: Rather than using simple interpolation, the algorithm applies a pre-calculated linear filter matrix to the reference samples. This operation computes an inner product between the chosen filter weights and the reference pixels, generating the predicted pixel values for the current 4x2 patch.
  4. State Update (Recursion): The rightmost column and bottom row of the newly predicted 4x2 patch are fed into the prediction buffer, acting as virtual reference boundaries for subsequent adjacent patches.
  5. Iterative Propagation: The algorithm moves through the block in raster or diagonal scan order. Each subsequent patch is computed using either external block boundaries, the internal synthetic boundaries created by previously processed patches, or a combination of both. This repeats until the entire block is populated.

Filter Modes and Weight Derivation

To keep encoding and decoding computationally feasible, AV1 avoids deriving filter weights dynamically during runtime. Instead, experimental and standard recursive intra modes utilize a fixed set of predefined filter kernels stored in lookup tables.

These filters are trained offline using machine learning techniques over large corpuses of video data. AV1 typically supports several distinct filter variations representing directional and non-directional characteristics, including:

During the rate-distortion optimization (RDO) search, the encoder tests these predefined filter modes against standard directional modes and signals the optimal mode index in the bitstream.

Advantages and Architectural Trade-offs

Recursive filtering allows boundary details to propagate naturally into the interior of a block, producing more realistic gradients and localized textures than straight linear projection. This reduces the energy of the residual signal, resulting in lower bitrate requirements for complex scenes.

However, the sequential nature of recursive filtering introduces intra-block data dependencies. Because patch \(N+1\) cannot be calculated until patch \(N\) finishes its matrix multiplication and updates the reference buffer, hardware decoders face serialization constraints, making internal parallelization within a single block more difficult compared to traditional directional projection.