How CDEF Directional Search Prevents AV1 Ringing

The Constrained Directional Enhancement Filter (CDEF) in the AV1 video codec eliminates edge-ringing artifacts by dynamically aligning its filtering operations with the native orientation of visual edges. Ringing artifacts typically arise when coarse transform quantization truncates high-frequency data near sharp contrast boundaries, producing unwanted ripples known as the Gibbs phenomenon. By performing a localized directional search, CDEF identifies the dominant angle of an edge within small pixel blocks and confines low-pass filtering along that trajectory, smoothing compression noise without crossing and blurring sharp structural lines.

CDEF operates on reconstructed frames immediately following the deblocking filter. To determine how to filter a region without degrading detail, the algorithm divides the frame into small, non-overlapping blocks—typically 8x8 pixels for analysis, which are further evaluated as 4x4 sub-blocks.

For each block, the directional search evaluates eight discrete angular directions, spanning from 0° to 157.5° in 22.5° increments. The search algorithm assesses each direction by calculating the variance of pixel intensities along lines parallel to the chosen angle. The direction that minimizes the variance along its directional lines—or conversely, maximizes the directional correlation—is selected as the dominant edge direction. By identifying this angle mathematically, the codec discovers the orientation where pixel values are naturally the most uniform.

Directionally Aligned Filtering

Once the primary direction is established, CDEF applies a non-separable, non-linear spatial filter tailored to that specific angle:

  1. Primary Taps Along the Edge: The primary filter taps are aligned directly along the detected orientation. Because pixel intensities naturally vary minimally along an edge, aggressive smoothing can occur parallel to the boundary without softening the edge itself. This clears the high-frequency ripples caused by quantization.
  2. Secondary Taps Across the Edge: Secondary filter taps are placed at a 45-degree offset to the primary direction to provide cross-directional stability. These taps use much smaller filter weights and strict clipping thresholds to prevent strong filtering across the edge boundary.

The "Constrained" Thresholding Mechanism

Directional alignment alone is not always enough to prevent blurring; complex textures or weak edges can cause false directional estimates. CDEF prevents artifact propagation through a conditional thresholding mechanism.

The filter computes the difference between the target pixel and neighboring tap pixels. If the difference is below a signaled threshold, the neighbor pixel is considered part of the same surface and smoothed. If the difference exceeds the threshold—indicating a true, sharp edge transition—the filter clips the impact of that tap.

By combining rigorous directional search with strictly constrained non-linear clipping, CDEF removes ringing artifacts along the edge path while completely preserving the crisp contrast that defines visual sharpness in compressed AV1 frames.