AV1 Compound Inter-Intra Prediction Explained

Compound inter-intra prediction is an advanced coding tool in the AV1 video codec that combines temporal motion compensation with spatial intra prediction inside a single block. By blending an inter-predicted reference with an intra-predicted reference using specialized weighting masks, AV1 significantly reduces residual energy in complex scenes such as lighting variations, moving shadows, and occlusion boundaries. This article explains the fundamentals of compound inter-intra prediction and details the specific mechanisms AV1 uses to blend these signals together.

What is Compound Inter-Intra Prediction?

In traditional video coding, an encoder must choose between two primary prediction types for any given block:

• Inter prediction: Predicts pixels using motion vectors pointing to previously decoded frames (temporal redundancy).
• Intra prediction: Predicts pixels using adjacent, already-reconstructed boundary pixels within the same frame (spatial redundancy).

While standard compound prediction combines two inter candidates (from different reference frames or motion vectors), AV1 introduces compound inter-intra prediction. This technique allows a block to simultaneously use one inter prediction signal and one intra prediction signal. It is particularly effective when motion compensation captures the general displacement of an object, but spatial neighbors provide better localized detail, such as at the edges of moving objects or during gradual scene transitions.

Allowed Intra Modes

To limit signaling overhead and computational complexity, AV1 restricts the intra prediction modes that can be paired with an inter signal. An encoder can combine an inter candidate with one of four basic intra modes:

1. DC Mode: Applies an average flat value derived from neighboring pixels.
2. Vertical Mode: Projects the reconstructed top row vertically downward.
3. Horizontal Mode: Projects the reconstructed left column horizontally across.
4. Smooth Mode: Uses bilinear interpolation to generate a gradual spatial gradient from the boundaries.

How AV1 Blends the Blocks

Once the inter predictor (\(P_{inter}\)) and intra predictor (\(P_{intra}\)) are generated, AV1 blends them on a pixel-by-pixel basis using a weighted mask (\(m\)). The final sample value (\(P\)) at coordinate \((x, y)\) is calculated as:

\[P(x, y) = m(x, y) \cdot P_{intra}(x, y) + (1 - m(x, y)) \cdot P_{inter}(x, y)\]

AV1 determines the mask values \(m(x, y)\) using two primary blending strategies:

1. Smooth Blending (Distance-Based Mask)

In smooth blending mode, the influence of the intra prediction decays as the distance from the reconstructed boundary increases.

Because intra prediction relies on spatial neighbors directly above and to the left of the current block, its accuracy is highest immediately adjacent to those boundaries and degrades further into the block. AV1 applies a predefined, 1D or 2D lookup table that assigns higher weights to \(P_{intra}\) near the top and left edges. As \((x, y)\) moves toward the bottom-right of the block, the weight shifts almost entirely to \(P_{inter}\).

2. Wedge-Based Blending

Wedge-based blending partitions the block into two regions using a codebook of predefined geometric shapes (wedges).

AV1 defines a set of 16 wedge orientations and offsets per block size. These wedges divide the block along straight lines at various angles (horizontal, vertical, or oblique). Along the dividing line, the codec applies a softened, anti-aliased transition zone rather than a sharp binary boundary. Pixels on the intra-designated side of the wedge receive a high intra weight, pixels on the opposite side receive a high inter weight, and pixels falling within the transition margin are smoothly linearly interpolated.

Normalization and Residual Encoding

The weights in both smooth and wedge masks are normalized to sum to a fixed power-of-two constant (typically 64) to allow fast fixed-point arithmetic without division:

\[P(x, y) = \left( m(x, y) \cdot P_{intra}(x, y) + (64 - m(x, y)) \cdot P_{inter}(x, y) + 32 \right) \gg 6\]

After blending, the resulting hybrid block \(P\) is subtracted from the original source block to create the residual signal. Because the blended block accounts for both motion displacement and boundary continuity, the residual typically contains far less high-frequency data, allowing the transform and quantization stages to achieve superior compression ratios compared to using pure inter or pure intra prediction alone.