Intra-Boundary Filtering in AV1 Video Coding

Intra-boundary filtering in the AV1 video codec is a prediction refinement technique designed to reduce edge artifacts and improve compression efficiency during intra-frame coding. By conditioning the reference samples along the top and left boundaries of a block before or during prediction, AV1 prevents artificial sharp edges and high-frequency discrepancies from propagating into the predicted block. This article explains how intra-boundary filtering functions, the mechanisms it uses to smooth reference edges, and why it is vital for modern video compression.

The Purpose of Intra-Boundary Filtering

In intra-frame prediction, a video codec predicts the contents of a target block using the reconstructed pixels from immediately adjacent, previously decoded blocks—specifically those above and to the left. Because these reference pixels contain quantization noise and abrupt transitions, using them raw can cause unnatural directional banding, ringing artifacts, or visible block boundaries. Intra-boundary filtering resolves this issue by applying filtering algorithms directly to the reference samples or blending boundary pixels smoothly into the predicted area.

Pre-Prediction Reference Smoothing

Before directional intra prediction angles are projected across a block, AV1 evaluates the reference sample arrays (the row above and the column to the left). To prevent noise propagation, AV1 employs adaptive reference smoothing:

1. Variance and Smoothness Detection: The encoder evaluates whether the boundary reference pixels form a smooth gradient or contain genuine high-contrast edges. If a natural, sharp texture is detected, heavy filtering is bypassed to preserve detail.
2. Low-Pass Filtering: When smoothing is beneficial, a low-pass filter—typically a 3-tap filter with coefficients like [1, 2, 1] / 4 or a 5-tap filter for larger block sizes—is applied to the reference arrays. This eliminates high-frequency noise from the reference line prior to sample projection.

Position-Dependent Prediction Combination (PDPC)

Beyond standard pre-filtering of reference arrays, AV1 incorporates Position-Dependent Prediction Combination (PDPC). PDPC operates directly at the boundary interface by blending the directional intra prediction with the neighboring reconstructed boundary samples.

• Distance-Weighted Blending: For every pixel inside the current block, PDPC calculates a weighted average between the directional predictor and the boundary pixels located directly above and to the left.
• Decay Function: The influence of the boundary pixels decays exponentially as the distance from the block boundary increases. Pixels in the first row or column receive the strongest boundary correction, while interior pixels rely almost entirely on the directional prediction.
• Transition Softening: This mechanism smooths the abrupt boundary step between the newly predicted block and its already reconstructed neighbors, ensuring continuity across block borders.

Benefits on Coding Efficiency

Smoothing reference edges directly affects the residual signal—the difference between the original video frame and the predicted block. Discontinuities and noisy reference borders introduce high-frequency components that require a substantial bitrate to encode via discrete cosine transforms (DCT) or asymmetric discrete sine transforms (ADST). By smoothing reference edges and boundary interfaces, intra-boundary filtering minimizes residual energy, allowing AV1 to preserve higher visual fidelity at significantly lower bitrates.