How ADST Improves Edge Coding in AV1

The AV1 video codec incorporates the Asymmetric Discrete Sine Transform (ADST) alongside the traditional Discrete Cosine Transform (DCT) to significantly optimize compression efficiency near block boundaries. In intra-prediction, pixel values are predicted using previously decoded neighboring blocks to the top and left, which creates a residual error that is asymmetric—typically near zero at the reference boundary and increasing farther away. By matching this non-uniform residual profile, the ADST minimizes high-frequency coefficients, suppresses edge artifacts, and significantly reduces the bitrate required to encode sharp boundaries.

In conventional video codecs like H.264 and standard HEVC, the Discrete Cosine Transform (DCT-2) serves as the primary tool for spatial energy compaction. The DCT operates under the assumption of even symmetry at block borders, implying that the statistical distribution of residual errors is uniform across the entire transform block. However, directional intra-prediction breaks this symmetry. Because predictors directly extrapolate from adjacent pixels, the prediction error directly adjacent to the reference edge is virtually zero, while prediction accuracy degrades toward the opposite side of the block. Forcing a symmetric transform like the DCT onto an asymmetric residual distribution introduces high-frequency components that require more bits to encode and often generate visible ringing artifacts around edges.

The ADST resolves this physical limitation by relying on sine basis functions characterized by odd symmetry. In continuous mathematics, a sine wave naturally approaches zero at its origin. The discrete variant used in AV1 is tailored so that the transform basis vectors inherently force a near-zero boundary condition at the predicted edge, while allowing the opposite edge to vary freely without artificial boundary constraints. This directly mirrors the physical behavior of intra-prediction residuals.

AV1 expands this functionality by implementing separable, hybrid transform combinations. Because a block has both a horizontal and a vertical component, AV1 can mix and match transform types across dimensions:

• 1D ADST paired with 1D DCT: Ideal for blocks with unidirectional prediction, such as pure horizontal or vertical prediction modes where only one axis exhibits boundary error decay.
• ADST in both directions (ADST + ADST): Used for diagonal intra-prediction modes where both top and left boundaries serve as immediate reference points.
• FlipADST: Reverses the spatial orientation of the transform to account for prediction modes that extrapolate from the bottom or right boundaries, ensuring the zero-error constraint remains anchored to the correct reference edge.

By accurately modeling the decaying nature of boundary residuals, the ADST concentrates the majority of signal energy into fewer low-frequency transform coefficients. This compact energy representation allows the AV1 quantizer to discard high-frequency noise without blurring sharp edge transitions, providing crisper edges, fewer boundary discontinuities, and superior overall rate-distortion performance.