How AV1 Uses Wiener Filters in Loop Restoration

The AOMedia Video 1 (AV1) codec employs a Loop Restoration tool to reverse blur and distortion introduced during the lossy transform and quantization stages. A key component of this tool is the separable Wiener filter, which models image degradation as a linear blur combined with additive noise to reconstruct fine details. This article explains how the Wiener filter functions within the AV1 Loop Restoration stage, covering its mathematical foundation, structural optimization, and frame-level execution.

The Role of Loop Restoration in AV1

In the AV1 decoding pipeline, Loop Restoration is the final in-loop filtering stage, positioned directly after the Deblocking Filter and the Constrained Directional Enhancement Filter (CDEF). While the deblocking filter eliminates grid discontinuities at block edges and CDEF suppresses directional ringing artifacts, they often smooth out subtle textures.

Loop Restoration compensates for this loss by acting as a deconvolution filter. For each designated region, the encoder can select between three options:

• No filtering
• Self-guided restoration
• A separable Wiener filter

Mathematical Principle of the Wiener Filter

The Wiener filter operates under the principle of Minimum Mean Squared Error (MMSE). It calculates a set of linear filter coefficients that minimize the expected squared difference between the restored pixel values and the original source pixels.

During encoding, the encoder analyzes both the original uncompressed source frame and the degraded, post-CDEF reconstructed frame. It computes:

1. The auto-correlation of the degraded frame.
2. The cross-correlation between the original and degraded frames.

Using these statistical matrices, the encoder solves the Wiener-Hopf equations to derive the optimal filter tap values. These coefficients are then quantized and transmitted in the bitstream to the decoder.

2D Separability and Filter Symmetry

A standard two-dimensional non-separable filter requires significant computation and bitstream overhead. AV1 reduces both complexity and transmission costs through two optimizations:

1. Separability: The 2D filtering operation is decoupled into two consecutive 1D passes—a horizontal filtering pass followed by a vertical filtering pass. Instead of calculating an \(N \times N\) matrix, the decoder applies an \(N\)-tap 1D filter horizontally, followed by an \(M\)-tap 1D filter vertically.
2. Symmetry: AV1 employs a 7-tap symmetric filter for both directions (often denoted as a \(7 \times 7\) separable filter). Due to horizontal and vertical symmetry, only three coefficients per direction need to be transmitted:
   • For a 7-tap filter \([h_3, h_2, h_1, h_0, h_1, h_2, h_3]\), the center tap \(h_0\) is constrained to maintain unity gain (\(\sum h_i = 1\)).
   • Consequently, the filter only requires signaling three independent parameters (\(h_1, h_2, h_3\)) per dimension.

This reduces the total signaled parameters for a \(7 \times 7\) filter from 49 down to just 6 values (3 horizontal and 3 vertical).

Restoration Units and Bitstream Signaling

Filtering is not applied uniformly across an entire frame. Instead, frames are partitioned into Restoration Units (RUs), which are independent square areas typically sized at \(64 \times 64\), \(128 \times 128\), or \(256 \times 256\) pixels, depending on frame resolution.

• Coefficient Sharing: Filter coefficients can be signaled globally for the entire frame, or individual coefficients can be assigned to different restoration units using entropy-coded deltas.
• Switching Mechanism: Each RU indicates whether it applies the Wiener filter, switches to the self-guided filter, or disables restoration entirely based on Rate-Distortion Optimization (RDO) at the encoder.

Fixed-Point Implementation and Boundary Handling

To ensure deterministic output across all hardware architectures, AV1 defines Wiener filtering using fixed-point integer arithmetic rather than floating-point math:

• Filter coefficients are quantized into 7-bit signed integers.
• Intermediate results from the horizontal pass are rounded and clamped to a defined bit-depth before entering the vertical pass.
• Pixel boundaries at the edges of frames or tiles are handled via edge extension (padding by repeating the boundary pixels) to prevent visual artifacts and avoid accessing undefined memory locations.