What Is Directional Smoothing in AV1 Intra Prediction?

Directional smoothing in AV1 intra prediction is an adaptive reference-filtering technique designed to enhance the quality of spatially predicted video blocks. This article breaks down how AV1 uses directional smoothing, the specific artifacts it prevents, how the filtering mechanism operates on reference samples, and its impact on overall video compression efficiency.

Intra Prediction in AV1

Intra prediction compresses video frames by predicting pixel values within a target block using previously decoded pixels from adjacent blocks (typically above and to the left). While older codecs like VP9 supported only 8 directional modes, AV1 significantly expands this capability to 56 directional intra prediction angles. These angles allow the encoder to project boundary pixels across the block with high geometric precision.

The Role of Directional Smoothing

When reference pixels are projected across a block at arbitrary angles, sharp discontinuities, noise, or step-like patterns in the reference boundary can propagate through the entire predicted block. This propagation leads to visual artifacts such as aliasing, false edges, or unnatural banding.

Directional smoothing solves this problem by applying a low-pass filter to the reference samples before they are projected along the prediction path. By smoothing out high-frequency noise and sudden pixel transitions in the reference line, the codec generates a continuous, natural gradient across the block.

How Directional Smoothing Works

Directional smoothing operates conditionally based on the chosen prediction mode and block characteristics:

  1. Angle and Mode Evaluation: The encoder determines whether smoothing is beneficial based on the prediction angle and block dimensions. Steeper or non-orthogonal angles often require stronger filtering because fractional pixel interpolation can amplify boundary discontinuities.
  2. Filter Selection: AV1 employs distinct smoothing filters, typically utilizing 2-tap or 3-tap low-pass filters (such as a [1, 2, 1] / 4 filter). The choice of filter depends on the degree of smoothing required.
  3. Adaptive Enablement: Directional smoothing is not applied uniformly. If a boundary edge contains legitimate, sharp structural information that matches the prediction angle, excessive smoothing could blur the edge. AV1 therefore uses thresholding to decide whether smoothing will reduce the prediction residual or degrade edge sharpness.

Impact on Compression Performance

By eliminating artificial edges in the prediction signal, directional smoothing reduces the energy contained in the residual data—the difference between the original block and the predicted block. Smaller residual values require fewer bits to encode during the transform and quantization stages, directly translating to improved rate-distortion (RD) performance and higher visual quality at lower bitrates.