AV1 Directional Prediction Interpolation Equations

This article provides an overview of the directional intra-prediction interpolation equations used for diagonal modes in the AV1 video codec. In AV1, directional intra prediction extends boundary reference pixels across a target coding block along specific angular trajectories. For diagonal orientations—such as 45-degree (D45), 135-degree (D135), and 203-degree (D203) modes—predicted values often fall at fractional sub-pixel positions. The following sections explain the exact mathematical formulas used to project coordinates, determine fractional shifts, and compute interpolated sample values.

Coordinate Projection and Step Sizes

Directional prediction projects target block coordinates \((x, y)\) onto the adjacent top or left reference sample arrays. Diagonal modes utilize predetermined angular step parameters, referred to as the directional derivative or step size \(d\):

• For vertical-like diagonal modes (e.g., nominally 45° and 135°): \[p = (x + 1) \cdot d\]
• For horizontal-like diagonal modes (e.g., nominally 203°): \[p = (y + 1) \cdot d\]

Here, \(p\) represents the projected position relative to the reference array origin, scaled to fixed-point integer precision. In standard AV1 directional modes, \(d\) corresponds to the fractional rate of pixel displacement along the boundary per block unit.

Sub-Pixel Index and Fractional Shift Derivation

AV1 evaluates directional sub-pixel positions at 1/32-sample resolution using a 6-bit fixed-point scaling factor. The projected position \(p\) is decomposed into an integer reference index (base) and a 5-bit fractional offset (shift):

\[\text{base} = p \gg 6\]

\[\text{shift} = (p \gg 1) \&\ 31\]

• \(\text{base}\) identifies the integer sample location within the boundary reference buffer.
• \(\text{shift}\) provides the fractional sub-pixel offset in the range of \([0, 31]\), representing thirty-secondths (\(1/32\)) of a sample distance between reference pixels.

Two-Tap Linear Interpolation Formula

When the fractional offset \(\text{shift}\) is zero, the prediction directly inherits the integer reference sample value:

\[\text{Pred}(x, y) = \text{Ref}[\text{base}]\]

When \(\text{shift} > 0\), AV1 applies a 2-tap linear interpolation equation between the two adjacent reference samples, \(\text{Ref}[\text{base}]\) and \(\text{Ref}[\text{base} + 1]\):

\[\text{Pred}(x, y) = \text{Round2}\Big((32 - \text{shift}) \cdot \text{Ref}[\text{base}] + \text{shift} \cdot \text{Ref}[\text{base} + 1],\; 5\Big)\]

The rounding operation \(\text{Round2}(v, n)\) is mathematically defined as:

\[\text{Round2}(v, n) = (v + (1 \ll (n - 1))) \gg n\]

For \(n = 5\), this resolves to:

\[\text{Pred}(x, y) = \Big((32 - \text{shift}) \cdot \text{Ref}[\text{base}] + \text{shift} \cdot \text{Ref}[\text{base} + 1] + 16\Big) \gg 5\]

Reference Array Upsampling (Pre-Interpolation)

For diagonal modes with strong directional orientations, AV1 may enable reference upsampling to minimize interpolation blur and edge artifacts. When active, reference boundaries are upsampled by a factor of 2 prior to projection using a 5-tap low-pass filter:

\[\text{Ref}_{\text{up}}[2i] = \text{Ref}[i]\]

\[\text{Ref}_{\text{up}}[2i + 1] = \text{Round2}\Big(-\text{Ref}[i - 1] + 9 \cdot \text{Ref}[i] + 9 \cdot \text{Ref}[i + 1] - \text{Ref}[i + 2],\; 4\Big)\]

Equivalent to:

\[\text{Ref}_{\text{up}}[2i + 1] = \Big(-\text{Ref}[i - 1] + 9 \cdot \text{Ref}[i] + 9 \cdot \text{Ref}[i + 1] - \text{Ref}[i + 2] + 8\Big) \gg 4\]

When upsampling is applied, the directional projection equations use the expanded array \(\text{Ref}_{\text{up}}\), effectively increasing directional accuracy and edge continuity along diagonal prediction paths.