AV1 Transform Syntax: Zigzag vs Col-Scan Modes

In the AV1 video codec, scanning orders determine how two-dimensional arrays of quantized transform coefficients are linearized into a one-dimensional sequence for entropy coding. This article examines the primary differences between the Zigzag and Col-Scan modes within AV1 transform syntax, detailing how their distinct coefficient traversal paths optimize coding efficiency based on the directional characteristics of the residual signal.

The primary difference between Zigzag and Col-Scan modes lies in the geometric path used to read coefficients from the transform matrix, which directly affects the clustering of non-zero values and the placement of the End-of-Block (EOB) marker during entropy coding.

Traversal Direction and Pattern

• Zigzag Scan: Traverses the coefficient block diagonally, moving back and forth between rows and columns from the top-left (DC component) toward the bottom-right (high-frequency AC components). It assumes that energy decreases roughly symmetrically along both the horizontal and vertical frequency axes.
• Col-Scan (Column Scan): Traverses the transform block primarily along the vertical axis, reading coefficients column by column (or in column-oriented bands) before advancing horizontally. This prioritizes vertical frequency components over horizontal ones.

Energy Distribution and Transform Pairing

AV1 adapts its scan order to match the selected 2D transform type (tx_type) and the underlying intra prediction direction:

• Zigzag Suitability: Zigzag is the standard default for symmetric transforms, such as 2D Discrete Cosine Transform (DCT_DCT), where the prediction residual has no strong directional bias. Because spatial correlation is isotropic, the highest-energy non-zero coefficients naturally cluster near the DC coordinate and taper off diagonally.
• Col-Scan Suitability: Col-Scan is deployed when the residual exhibits strong anisotropic energy distribution—frequently occurring with asymmetric transform pairs (such as combinations of DCT, Asymmetric Discrete Sine Transform [ADST], or Identity transforms) or horizontal intra prediction modes. When spatial patterns vary heavily across vertical rows while remaining uniform horizontally, energy concentrates deeper along the vertical frequency axis.

Impact on Entropy Coding Efficiency

The AV1 syntax relies on grouping non-zero coefficients as early as possible in the 1D scan array. By matching the scan order to the coefficient energy distribution:

1. Earlier EOB Signaling: Col-Scan groups non-zero vertical frequencies ahead of zeroed horizontal frequencies. This ensures that trailing zeros appear earlier in the stream, allowing the encoder to signal the EOB marker sooner and truncate the coefficient stream.
2. Context Modeling: AV1’s arithmetic coder uses neighbor-based context models that depend on previously coded coefficients in scan order. Using Col-Scan for vertically dominant residuals ensures probability models accurately reflect the likelihood of consecutive non-zero coefficients along columns.