How ADST Improves Edge Encoding in AVIF Images

The AVIF image format, derived from the AV1 video codec, achieves superior compression efficiency partly through advanced transform techniques like the Asymmetric Discrete Sine Transform (ADST). This article explains how ADST overcomes the mathematical limitations of the standard Discrete Cosine Transform (DCT) to preserve sharp edges, reduce ringing artifacts, and improve the visual fidelity of intra-frame boundaries in AVIF images.

Traditional image formats like JPEG rely heavily on the symmetric Type-II Discrete Cosine Transform (DCT). While DCT is effective at compacting energy across smooth, continuous pixel blocks, it enforces an implied even symmetry at block boundaries. In intra-prediction—where a block's pixel values are predicted from previously decoded neighboring pixels along the top and left borders—the prediction error (residual) is not symmetric. Instead, the residual typically starts near zero at the predicted boundary and grows larger as the distance from the reference pixels increases. Forcing this non-symmetric residual into a symmetric DCT requires many high-frequency coefficients, leading to inefficient compression and visible ringing artifacts (the Gibbs phenomenon) around sharp borders and text.

The Asymmetric Discrete Sine Transform directly addresses this structural mismatch. Mathematically, the ADST enforces zero-boundary conditions at one end of the block (odd symmetry) while allowing natural variation at the other. Because prediction residuals naturally exhibit low amplitude near reference edges and expand across the block, the basis functions of the ADST naturally match the shape and decay profile of these directional prediction errors.

In AVIF encoding, this dynamic improves edge representation in several distinct ways:

• Superior Energy Compaction: Because the ADST basis functions closely align with the one-sided gradient of intra-prediction residuals, the residual energy is concentrated into fewer transform coefficients. This allows encoders to discard more high-frequency data without losing crucial structural details along an edge.
• Elimination of Ringing and Halos: Sharp edges often suffer from checkerboarding or "haloing" when encoded via symmetric transforms. By avoiding forced boundary mirroring, ADST models sharp step transitions and high-contrast lines without introducing artificial oscillations.
• Flexible 2D Separable Transform Combinations: AVIF does not apply ADST globally; it pairs 1D transforms independently across horizontal and vertical dimensions. An encoder can pair a 1D-DCT horizontally with a 1D-ADST vertically (or use flipped ADST variations) depending on the exact angle and direction of an edge.

By integrating the Asymmetric Discrete Sine Transform into its hybrid transform core, AVIF captures sharp contrasts, UI elements, vector-like graphics, and natural contours at significantly lower bitrates than codecs restricted to standard cosine transforms.