AV1 Lossless Mode: Mathematical Explanation

This article provides an overview of the lossless coding mode in the AOMedia Video 1 (AV1) codec, detailing the technical architecture that enables bit-exact reconstruction of source video. It explains how AV1 eliminates information loss by bypassing irreversible transforms and quantization, utilizing exact integer spatial predictions, and preserving absolute numerical precision through entropy coding.

What is Lossless Mode in AV1?

In standard video compression, codecs achieve high efficiency by discarding visually imperceptible details through lossy compression. AV1 lossless mode disables this data reduction mechanism, ensuring that the decoded output is numerically identical—bit for bit—to the original uncompressed input.

Lossless encoding is primarily used in archival workflows, medical imaging, screen content capture, and video editing pipelines where compression artifacts or generational loss cannot be tolerated.

The Lossy Video Pipeline vs. Lossless Mode

To understand how lossless mode works mathematically, it is necessary to examine the standard compression pipeline, which consists of four primary stages:

1. Prediction: Inter-frame (motion compensation) or intra-frame (spatial) prediction creates an estimate of the block.
2. Residual Calculation: The predicted block is subtracted from the original block to produce a residual.
3. Frequency Transform: Transforms (such as Discrete Cosine Transform or Asymmetric Discrete Sine Transform) convert spatial residuals into frequency coefficients.
4. Quantization: High-frequency coefficients are divided by a quantization step size and rounded to nearest integers, discarding information.
5. Entropy Coding: A symbol-level coder compresses the remaining quantized values losslessly.

In standard encoding, Quantization is the primary source of loss, while non-reversible floating-point approximations in the Transform stage can introduce secondary drift. AV1's lossless mode restructures this pipeline to ensure full reversibility.

Mathematical Implementation of AV1 Lossless Mode

Lossless compression in AV1 is triggered when the quantizer parameter is set to zero (qindex = 0), which enables a bit-exact integer processing path.

1. The Zero Quantization Step (\(Q_{step} = 1\))

Quantization is defined mathematically as:

\[q(x, y) = \text{round}\left(\frac{C(x, y)}{Q_{step}}\right)\]

Where \(C(x, y)\) is the coefficient and \(Q_{step}\) is the scale factor. Reconstruction in a lossy decoder approximates the coefficient:

\[\hat{C}(x, y) = q(x, y) \times Q_{step}\]

Information is permanently lost whenever \(C(x, y) / Q_{step}\) is non-integer.

In AV1 lossless mode, the quantization index is mapped such that the effective step size is exactly unity (\(Q_{step} = 1\)) with zero offset:

\[q(x, y) = C(x, y)\]

Because no division or rounding takes place, \(q(x, y)\) maintains exact numerical equality with the input coefficient \(C(x, y)\).

2. Transform Skip Mode (Spatial Domain Residuals)

Frequency transforms often introduce rounding errors due to irrational transform matrix constants. AV1 solves this in lossless mode by using Transform Skip.

Instead of applying a 2D transform matrix \(T\):

\[C = T \cdot R \cdot T^T\]

The transform step is replaced with the identity operation:

\[C(x, y) = R(x, y)\]

Where \(R(x, y)\) is the raw integer residual in the spatial domain. This avoids integer-rounding approximations and processes the spatial pixel differences directly.

3. Bit-Exact Prediction and Invertibility

AV1 calculates the prediction \(P(x, y)\) using neighboring reconstructed pixels via strictly defined integer arithmetic (such as directional intra-prediction or block matching).

The residual \(R(x, y)\) is computed via simple subtraction:

\[R(x, y) = S(x, y) - P(x, y)\]

Where \(S(x, y)\) is the original integer pixel value.

Because transform and quantization stages are identity operations, the residual decoded by the receiver \(\hat{R}(x, y)\) is mathematically identical to \(R(x, y)\):

\[\hat{R}(x, y) = R(x, y)\]

The decoder reconstructs the original pixel \(\hat{S}(x, y)\) by adding the prediction:

\[\hat{S}(x, y) = P(x, y) + \hat{R}(x, y)\]

Substituting the encoded residual into the reconstruction equation:

\[\hat{S}(x, y) = P(x, y) + (S(x, y) - P(x, y)) = S(x, y)\]

The prediction term cancels out entirely, guaranteeing that the reconstructed pixel \(\hat{S}(x, y)\) equals the source pixel \(S(x, y)\) without drift.

4. Entropy Coding via Daala Multi-Symbol Arithmetic Coder

Once spatial residuals are produced, they are compressed using AV1’s entropy coder—an advanced multi-symbol arithmetic coder derived from the Daala project.

Arithmetic coding represents a sequence of discrete integer symbols as a single fractional value within a defined range \([0, 1)\). Because arithmetic coding is a bijection (a one-to-one mapping between symbol sequences and encoded bit intervals), it compresses data strictly according to its statistical entropy without altering symbol values.

5. Color Space Considerations

True lossless encoding requires preserving the original color space. Standard video often converts RGB sources to YUV 4:2:0, introducing chroma subsampling loss before encoding begins.

In AV1 lossless mode, full 4:4:4 sampling or native RGB coding is typically used. When color space transformations are required to decorrelate color channels, AV1 uses Reversible Color Transforms (RCT) implemented with integer lifting steps:

\[Y = \lfloor (R + 2G + B) / 4 \rfloor\] \[U = R - G\] \[V = B - G\]

All operations are reversible using exact inverse integer arithmetic, preventing color degradation.