AV1 In-Loop Filters vs HEVC SAO Complexity

This article examines the computational complexity differences between the in-loop filtering architectures of the AV1 and HEVC (H.265) video codecs. While High Efficiency Video Coding (HEVC) relies on a lightweight two-stage filtering process featuring Sample Adaptive Offset (SAO), AV1 uses an advanced multi-stage pipeline that includes Constrained Directional Enhancement Filtering (CDEF) and Loop Restoration (LR). As a result, AV1 provides superior visual artifact removal and compression efficiency, but demands substantially higher computational resources, memory bandwidth, and silicon area than HEVC's SAO.

HEVC In-Loop Filtering and SAO Architecture

HEVC processes decoded pictures using a sequential two-step in-loop filtering pipeline: a traditional Deblocking Filter (DBF) followed by the Sample Adaptive Offset (SAO) filter.

SAO operates conditionally on reconstructed samples after the deblocking stage. Its primary purpose is to reduce ringing artifacts and reconstruct edge gradients by categorizing pixels and applying simple offsets. SAO operates in two basic modes:

• Edge Offset (EO): Compares a current pixel with two neighboring pixels along one of four predetermined directions (horizontal, vertical, or two diagonals) to classify it into one of five categories (local minimum, local maximum, two edge gradients, or flat).
• Band Offset (BO): Divides the sample amplitude range into 32 uniform bands and adds an offset to samples falling within four selected consecutive bands.

From a computational standpoint, SAO is exceptionally lightweight. The pixel classification requires basic integer comparisons, conditional branching, and simple additions. It requires minimal line buffer storage (typically only one line of pixels above the current Processing Tree Unit) and introduces very low latency and memory bandwidth consumption during both hardware and software decoding.

AV1 In-Loop Filtering Pipeline

AV1 replaces the simpler post-deblocking approach with a sophisticated, four-stage in-loop filtering system:

1. Deblocking Filter (DBF): Removes grid-like boundary artifacts using varied filter lengths.
2. Constrained Directional Enhancement Filter (CDEF): Applied directly after deblocking, CDEF identifies the primary directional orientation of edges within 8x8 blocks. It then applies a non-linear low-pass filter along the detected edge direction to eliminate ringing artifacts without blurring fine details.
3. Super-Resolution: An optional normative upscaling step applied inside the loop.
4. Loop Restoration (LR): Applied after CDEF, LR operates at a configurable block level (typically 64x64 to 256x256) using one of two mathematical algorithms:
   • Separable Symmetric Wiener Filter: A 7-tap separable filter designed to reverse blur and restore high frequencies.
   • Dual Self-Guided Filter: Employs two different edge-preserving box-filtering passes with adaptive linear regression weights to restore image details.

Computational Complexity Comparison

Comparing the computational footprint of AV1’s post-deblock filters (CDEF and Loop Restoration) against HEVC's SAO reveals sharp contrasts across several dimensions:

1. Arithmetic and Operational Intensity

• HEVC SAO: Consists primarily of conditional logic and small addition operations. The classification process requires only two comparisons per sample for Edge Offset, and a bit-shift/mask operation for Band Offset.
• AV1 In-Loop Filters: CDEF requires an initial search stage that computes directional variance across eight possible angles for every block, followed by non-linear clamping and multiply-accumulate operations along the dominant edge. Loop Restoration introduces higher complexity: the Wiener filter requires multi-tap 2D separable convolutions, while the Self-Guided filter requires integral image calculations, variance evaluations, and localized linear regressions.

2. Memory Access and Line Buffering

• HEVC SAO: Needs access to immediate neighboring pixels, requiring only a shallow line buffer (typically 1 line) to cross horizontal Processing Tree Unit (CTU) boundaries.
• AV1 In-Loop Filters: Requires multiple line buffers between stages. CDEF requires access to pixel margins around 8x8 blocks, and Loop Restoration requires access to up to three reconstructed lines above and below boundaries. This substantially increases on-chip SRAM requirements in dedicated hardware decoders.

3. Execution Overhead in Decoders

• HEVC SAO: Typically accounts for less than 3% to 5% of total decoder processing time on general-purpose CPUs.
• AV1 In-Loop Filters: CDEF and Loop Restoration combined routinely consume between 20% and 35% of total software decoding time unless aggressively vectorized (via AVX-512, AVX2, or ARM NEON).

Summary

The AV1 in-loop filter suite is significantly more computationally complex than the HEVC SAO filter. While HEVC prioritized ultra-low hardware implementation costs and simple arithmetic for SAO, AV1 trade-offs processing power and line-buffer memory to achieve higher structural fidelity, resulting in an in-loop filtering phase that is an order of magnitude heavier than that of HEVC.