AV1 Inter-Component Prediction: Y, U, and V

This article provides an overview of how the AV1 video codec leverages correlation between color channels to boost compression efficiency. Specifically, it explores Chroma from Luma (CfL) prediction, detailing the mathematical and structural methods AV1 uses to reconstruct the chroma (\(U\) and \(V\)) planes directly from the reconstructed luma (\(Y\)) plane.

The Role of Chroma from Luma (CfL)

In digital video compression, color is typically represented in the \(YCbCr\) (or \(YUV\)) color space. Human vision is significantly more sensitive to brightness (\(Y\)) than to color deviations (\(U\) and \(V\)). Consequently, the luma channel contains the sharpest edges, contours, and textures. Traditional video codecs often predict chroma blocks using directional intra modes independently from luma. AV1 introduces Chroma from Luma (CfL), an intra-prediction tool that allows the encoder to model chroma sample values directly as a linear function of the coincident reconstructed luma samples.

CfL operates strictly in one direction: from luma to chroma. AV1 does not perform prediction between the two chroma channels (\(U\) to \(V\) or vice versa), nor does it predict luma from chroma.

Alignment and Downsampling

Because video is commonly encoded in 4:2:0 subsampling, the chroma components have half the horizontal and vertical resolution of the luma plane. Before luma details can be used to predict chroma, they must be spatially aligned:

1. Reconstruction: The corresponding luma block is fully reconstructed (predicted, transformed, quantized, and inverse-transformed) before chroma prediction occurs.
2. Subsampling: The reconstructed luma block is downsampled to match the resolution of the target chroma block. AV1 uses standard box filtering or simple averaging depending on the subsampling format (e.g., averaging a \(2 \times 2\) block of luma samples to produce a single value matching a \(4:2:0\) chroma sample position).

The Linear Model Formulation

Once the luma component is downsampled, AV1 models the chroma prediction as a scaled version of the AC (alternating current, or detail) component of the luma signal, offset by an intra DC baseline.

The mathematical formulation for a chroma sample \(C(x, y)\) is:

\[C_{pred}(x, y) = DC_{chroma} + \alpha \cdot (L(x, y) - \mu_L)\]

Where:

• \(L(x, y)\) represents the downsampled reconstructed luma sample.
• \(\mu_L\) is the average (DC value) of the downsampled reconstructed luma block.
• \((L(x, y) - \mu_L)\) isolates the AC component, representing the edges and textures without the absolute brightness offset.
• \(\alpha\) is a scaling factor signaled in the bitstream.
• \(DC_{chroma}\) is the standard intra DC prediction calculated from neighboring reconstructed chroma pixels.

Parameter Signaling

To minimize bitrate overhead, the scaling factor (\(\alpha\)) is quantized and transmitted explicitly for each chroma plane (\(U\) and \(V\)) independently.

• The sign and magnitude of \(\alpha\) are coded into the bitstream.
• A value of \(\alpha = 0\) effectively turns off CfL, falling back to regular DC prediction.
• Signaling \(\alpha\) separately for \(U\) and \(V\) allows the codec to adapt to varying color relationships, such as when one chroma channel is positively correlated with brightness while the other is negatively correlated.

By relying on the already-transmitted texture details in the luma plane and only sending the scaling parameter \(\alpha\), AV1 eliminates redundant edge signaling across color planes and significantly reduces the bitrate required for chroma intra coding.