Compound Prediction Modes in AV1 Explained

Compound prediction in the AV1 video codec is an advanced inter-frame compression technique that combines two distinct reference predictors to reconstruct a single image block. By blending motion-compensated signals from multiple frames—or combining temporal and spatial predictions—AV1 significantly reduces residual data, lowering bitrate requirements while preserving high visual quality. This article breaks down the mechanics of compound prediction, its operational pipeline, and the specialized compound modes implemented in AV1.

What Is Compound Prediction?

In standard inter-frame video coding, uni-directional prediction relies on a single motion vector pointing to one reference frame. Compound prediction expands this by utilizing two separate motion vectors to retrieve two prediction blocks, usually from different reference frames (or different temporal directions). The encoder then blends these two signals together to form the final predicted block.

By combining two sources of visual data, the codec effectively averages out noise, handles lighting variations, and resolves motion inconsistencies far better than single-reference prediction.

Types of Compound Prediction Modes in AV1

AV1 significantly expands upon the traditional bi-directional prediction found in older standards like H.264 and HEVC. It introduces several distinct modes to manage complex scenes:

1. Averaging and Distance-Weighted Prediction
   The most straightforward compound mode applies a uniform average across the two prediction blocks. In distance-weighted mode, AV1 assigns weights to each predictor based on the temporal distance between the current frame and the respective reference frames, giving higher priority to the temporally closer frame.

2. Wedge-Based Prediction
   Wedge prediction partitions a coding block into two regions using a predefined codebook of geometric edge orientations (vertical, horizontal, or oblique lines). Rather than creating a hard boundary, AV1 applies a smooth blending mask along the wedge boundary. This mode excels when an edge of a moving foreground object cuts through a single block, allowing each side of the edge to track a different motion path.

3. Difference-Weighted Prediction (Masked Compound)
   In this mode, AV1 generates a pixel-by-pixel blending mask determined by the absolute pixel difference between the two reference predictors. Where the two predictors closely match, the codec relies on equal averaging; where they diverge, it favors the reference signal that aligns more consistently with surrounding spatial content. This adapts well to local luminance changes and soft boundaries.

4. Compound Intra-Inter Prediction
   Unlike traditional inter-only blending, AV1 can combine an intra (spatial) predictor and an inter (temporal) predictor within the same block. The spatial prediction is derived from neighboring reconstructed pixels in the current frame, while the temporal prediction comes from a reference frame. This mode is particularly useful for disocclusions, shadows, or rapid scene transitions where historical frame data is only partially reliable.

How Compound Modes Function in the AV1 Pipeline

The implementation of compound prediction follows a systematic process during block-level encoding:

• Reference Selection: The encoder evaluates its reference frame buffers—such as LAST_FRAME, BWDREF_FRAME, or ALTREF_FRAME—and selects an optimal pair of references along with their corresponding motion vectors.
• Predictor Generation: Two independent motion-compensated candidate blocks (\(P_1\) and \(P_2\)) are extracted using high-precision sub-pixel interpolation filters.
• Mask Creation: Depending on the selected mode (uniform, wedge, or difference-weighted), a weight matrix (mask \(M\)) is either looked up from a predefined codebook or computed dynamically.
• Signal Blending: The final prediction \(P\) is calculated at the pixel level using the formula:
  \(P = (M \times P_1 + (1 - M) \times P_2)\)
• Residual Transformation: The blended prediction is subtracted from the original block to produce the residual error. Because the compound prediction closely matches the source signal, this residual is minimal, allowing transform and quantization tools to compress it with high efficiency.

Compound prediction modes are a fundamental reason AV1 achieves substantial compression gains over predecessors like VP9 and H.264, particularly in complex sequences involving occlusions, complex camera motion, and dynamic lighting.