AV1 Deblocking: Luma vs Chroma Filter Tap Lengths

The AOMedia Video 1 (AV1) codec utilizes an adaptive in-loop deblocking filter designed to suppress grid-like boundary artifacts while preserving true image detail. The deblocking process treats luminance (luma) and chrominance (chroma) channels differently by applying distinct filter tap lengths to each. While the luma channel uses longer filter taps—ranging up to 14 taps—to smooth flat image regions, the chroma channels are constrained to shorter tap lengths—typically capped at 6 or 4 taps—to prevent color bleeding, preserve color contrast, and reduce overall computational complexity.

Luma Filter Tap Lengths

The luma (\(Y\)) plane contains the majority of the structural and high-frequency spatial details perceptible to the human eye. To handle different artifact intensities across block edges, AV1 provides three primary filter lengths for luma boundaries:

• 14-Tap Filter (Wide Filter): Applied in very smooth, flat regions across \(8\times8\) or larger block boundaries. It evaluates up to seven samples on each side of the block edge and can modify up to six samples on either side (a total footprint of 14 taps modifying 12 samples).
• 8-Tap Filter (Regular Filter): Applied when flat-region conditions are met over a narrower spatial span. It evaluates four samples on each side of the edge and can modify up to three samples on either side.
• 4-Tap Filter (Narrow Filter): Used for standard, high-detail, or smaller block edges (such as \(4\times4\) boundaries). It evaluates two samples on each side and modifies up to two samples on either side.

Chroma Filter Tap Lengths

The chroma (\(Cb\) and \(Cr\)) planes carry color information and are typically subsampled (most commonly in 4:2:0 format). Because chroma samples represent larger spatial areas, applying long filters would severely blur color boundaries. AV1 limits chroma deblocking to shorter tap lengths:

• 6-Tap Filter: Chroma's equivalent of an extended filter, used when adjacent blocks are relatively flat and the boundary permits deeper smoothing. It evaluates three samples on each side of the boundary and modifies up to two or three samples on each side.
• 4-Tap Filter: The standard filter for chroma boundaries. It evaluates two samples on each side of the edge and modifies up to two samples on either side.
• Absence of the 14-Tap Filter: Chroma deblocking explicitly disables the ultra-wide 14-tap filter used by the luma channel.

Summary of Differences

• Maximum Tap Length: Luma supports up to 14-tap filtering, whereas chroma is restricted to a maximum of 6 taps.
• Number of Modified Samples: Luma filtering can adjust up to 6 samples on each side of a transform or prediction block boundary. Chroma filtering modifies a maximum of 2 to 3 samples per side.
• Boundary Granularity: In 4:2:0 subsampling, chroma block boundaries occur at half the spatial frequency of luma boundaries. Consequently, chroma filtering operates on a coarser coordinate grid, necessitating smaller tap counts to prevent over-filtering.

Why the Differences Exist

1. Mitigation of Color Bleeding: Applying wide 14-tap filters to chroma components causes color to bleed across object edges, producing visible discoloration and blurring along distinct object contours.
2. Subsampling Geometry: In 4:2:0 video, one chroma sample corresponds spatially to a \(2\times2\) luma region. A 6-tap filter on a chroma plane already spans an equivalent spatial distance of a 12-tap filter on the luma plane.
3. Computational Efficiency: Chroma processing accounts for roughly one-third of the raw sample count in standard video profiles. Limiting tap sizes avoids complex branch checks and costly multi-tap arithmetic across color channels.