How AV1 CDEF Uses Primary and Secondary Taps

The Constrained Directional Enhancement Filter (CDEF) in the AV1 video codec removes ringing artifacts and preserves edge sharpness by filtering along the dominant direction of image features. By partitioning reconstructed frames into blocks, determining the primary signal orientation, and applying non-linear primary and secondary filter taps along and across that direction, CDEF prevents the blurring of genuine edges while smoothing out high-frequency compression noise.

Direction Search

Before any filtering occurs, CDEF determines the dominant orientation of edges within each 8x8 block (subdivided into 4x4 sub-blocks for processing). The encoder or decoder tests eight possible directions, spanning angles from 0° to 157.5° in 22.5° increments.

The algorithm calculates the variance of pixel values along lines following each candidate direction. The direction with the lowest variance—meaning the pixels along that line are the most uniform—is selected as the primary direction. If a block has low contrast or no discernible pattern, the algorithm identifies it as non-directional, and filtering adjusts accordingly.

Primary Filter Taps

Once the primary direction is established, CDEF aligns a 1D filter kernel directly along this axis. The primary filter taps sample two neighboring pixels on either side of the target pixel following the exact angle detected during the direction search.

Because these taps operate exclusively parallel to edges rather than across them, they smooth out ringing noise along lines and contours without blending foreground and background colors together. Smoothing parallel to an edge reduces coding artifacts while preserving the contrast and crispness of the boundary.

Secondary Filter Taps

To suppress artifacts that primary taps alone cannot eliminate, CDEF employs secondary filter taps. These taps form a cross-shaped pattern oriented at 45-degree angles relative to the detected primary direction.

The secondary taps pull samples from four additional neighbors located off the primary axis. Because these taps inevitably cross edge boundaries, they are configured with significantly lower filter weights and tighter constraints than the primary taps. This configuration allows them to reduce ringing in flatter surrounding regions without dulling the primary edge.

Non-Linear Thresholding and Clamping

The defining feature of CDEF is its conditional, non-linear clamping function, which dictates how the primary and secondary taps modify the target pixel:

\[\text{Pixel}_{\text{filtered}} = \text{Pixel} + \sum_{i} w_i \cdot f(\text{Neighbor}_i - \text{Pixel}, S, D)\]

Where:

• \(w_i\) represents the tap weight.
• \(S\) is the filter strength parameter (signaled separately for primary and secondary taps).
• \(D\) is a damping parameter configured at the frame level.
• \(f(d, S, D)\) is a non-linear threshold function that clamps the pixel difference.

If a neighboring pixel's value differs significantly from the target pixel—exceeding the threshold dictated by \(S\) and \(D\)—the function clamps its contribution to zero or near zero. This prevents out-of-boundary details from bleeding into the target pixel.

By running non-linear primary taps along the edge and secondary taps at diagonal offsets, CDEF achieves targeted smoothing where signals correlate, while immediately cutting off filter weights where genuine object boundaries exist.