How CDEF Detects Edges to Prevent Blurring in AV1

The Constrained Directional Enhancement Filter (CDEF) is a core in-loop filtering tool in the AV1 video codec designed to eliminate ringing artifacts without sacrificing edge sharpness. By combining the directional deblocking filter from the Thor codec with the deringing filter from the Daala codec, CDEF analyzes block textures to locate dominant edges and aligns its filtering along those lines rather than across them. This article explains the exact mechanism CDEF uses to identify directional edges and adapt its filtering to preserve vital video details.

Block-Level Direction Search

CDEF analyzes reconstructed video frames in small blocks, typically divided into 8x8 pixel regions (which may be subdivided into 4x4 blocks for chrominance). Because ringing artifacts are most noticeable around high-contrast transitions, the filter must first determine if a dominant edge exists within each block and determine its precise angle.

To do this, CDEF evaluates eight discrete directions, representing angles spaced at roughly 22.5-degree intervals (\(0^\circ, 22.5^\circ, 45^\circ, 67.5^\circ, 90^\circ, 112.5^\circ, 135^\circ\), and \(157.5^\circ\)).

Directional Variance Calculation

The algorithm identifies the correct direction by measuring pixel variance along lines parallel to each candidate angle. For each of the eight directions:

1. The block's pixels are grouped into lines oriented along that specific angle.
2. The sum of pixel intensities along each parallel line is calculated.
3. The algorithm computes the variance of these line sums across the block.

When a candidate direction perfectly matches an edge, the pixel values along each individual line remain relatively uniform, but the contrast between parallel lines is maximized. Consequently, the direction that produces the highest variance among its line averages is chosen as the dominant edge direction.

Along with the best direction, the algorithm calculates the directional strength. If the maximum variance is low, the block is classified as flat or textureless, meaning no strong edge is present.

Direction-Aligned Non-Linear Filtering

Once the primary direction is established, CDEF applies a non-linear conditional filter designed specifically to avoid blurring:

• Primary Filter: Aligned strictly along the detected edge direction. Because it smooths pixels along the path of the edge rather than across it, it eliminates ringing without softening the transition between distinct visual regions.
• Secondary Filter: Configured at a 45-degree cross-direction to catch residual noise. It uses significantly lower strength thresholds to prevent degradation of the main edge.

Thresholding to Protect Detail

To guarantee that genuine high-frequency details are not erased, CDEF uses clipping thresholds (primary and secondary strengths) signaled at the frame or block level. If the difference between a neighboring pixel and the target pixel exceeds the threshold, the neighbor's contribution is clamped. This ensures that the filter only smooths low-amplitude ringing artifacts while leaving large, structurally significant pixel transitions untouched.