AV1 Deblocking Filter Edge Thresholds Explained

The AV1 video codec employs an in-loop deblocking filter to eliminate blocking artifacts caused by transform and prediction block partitions while preserving true image edges. To prevent the unwanted blurring of genuine picture details, the filter evaluates local pixel gradients across block boundaries against mathematically defined edge sharpness thresholds. These thresholds are derived from a combination of frame-level quantization parameters, signaled filter levels, delta adjustments, and a dedicated sharpness control setting.

Base Filter Level Derivation

The foundation of threshold calculation begins at the frame level. The encoder signals global filter levels for the luma and chroma components, distinguishing between vertical and horizontal block edges. These base levels are closely coupled with the frame’s Quantization Parameter (QP). A higher QP signifies higher quantization error, which generally requires higher filter levels to smooth coarser block boundaries. Conversely, lower QP values yield lower filter levels to keep fine textures intact.

AV1 can refine these base levels locally across the frame using:

• Segment Deltas: Adjustments specific to predefined image segments.
• Filter Level Deltas: Offsets applied based on whether a block is an intra- or inter-coded block, as well as the specific reference frame utilized.

Boundary Thresholds: Limit, Blimit, and Thresh

Once the final filter level is calculated for a given block boundary, the decoder computes three primary threshold parameters:

1. limit: Controls the maximum allowable variation among internal samples on either side of the boundary.
2. blimit (Block Limit): Regulates the acceptable step difference directly across the boundary edge.
3. thresh (High Edge Variance): Detects high-frequency variation within samples on one side of the edge to determine whether a narrow (weak) or wide (strong) filter is applied.

The blimit threshold is proportional to the filter level, scaled to evaluate the gradient step between adjacent pixels across the edge. The limit threshold is derived by modifying the filter level according to the frame's global sharpness parameter.

The Role of the Sharpness Parameter

AV1 features a filter_sharpness parameter (ranging from 0 to 7) that shifts the threshold values to favor detail retention over artifact suppression:

• When filter_sharpness is set to 0, the threshold calculation relies purely on the base filter level, providing standard smoothing.
• As filter_sharpness increases, the filter systematically scales down the limit threshold.

Reducing limit restricts the range of pixel gradients that qualify for smoothing. If a local gradient exceeds this reduced limit, the algorithm classifies the transition as an authentic image detail rather than a compression artifact, thereby skipping or dampening the filtering process.

Boundary Condition Testing

During filtering, pixels perpendicular to the edge (designated as \(p\) samples on one side and \(q\) samples on the other, numbered outward from the boundary as \(p_0, p_1, p_2...\) and \(q_0, q_1, q_2...\)) are tested against the calculated thresholds:

• Edge Filter Condition: Filtering only occurs if the gradient across the boundary satisfies: \[2 \times |p_0 - q_0| + \frac{|p_1 - q_1|}{2} < blimit\] and the inner gradients satisfy: \[|p_1 - p_0| < limit \quad \text{and} \quad |q_1 - q_0| < limit\]
• High-Edge Variance (HEV) Test: If the edge passes the initial condition, the filter checks whether \(|p_1 - p_0| > thresh\) or \(|q_1 - q_0| > thresh\). If either is true, high variance is detected near the edge, restricting the operation to a weak filter that only modifies the immediate boundary samples (\(p_0, q_0\)) to avoid smearing the neighboring detail.

Through this multi-tiered process of QP scaling, sharpness shifting, and local sample testing, AV1 dynamically determines sharpness thresholds that maintain a balance between eliminating block seams and retaining authentic texture.