ARM NEON Optimization for AV1 Interpolation Filters

This article explores how the ARM NEON SIMD architecture accelerates the computationally intensive 7-tap and 8-tap sub-pixel interpolation filters used in AV1 video decoding and encoding. It details how NEON handles the separable horizontal and vertical filtering stages through vector permutation, efficient register reuse, widening arithmetic, and rounding instructions to maximize throughput while maintaining strict bit-exactness with the AV1 specification.

The Role of 7-Tap and 8-Tap Filters in AV1

AV1 motion compensation relies on sub-pixel precision to estimate motion vectors between reference and target frames. Fractional pixel positions are calculated using separable filtering: a horizontal 1D pass followed by a vertical 1D pass (or vice versa).

AV1 utilizes a family of 8-tap and 7-tap filters (such as regular, smooth, and sharp filters) depending on the sub-pixel offset and prediction mode. The 8-tap filter requires eight consecutive source samples to produce a single interpolated output sample, while the 7-tap variation drops one edge tap, effectively treating it as a specialized 8-tap filter with a zero coefficient or an asymmetric profile. Because these filters must process large arrays of video data at high frame rates, naive scalar implementations introduce severe bottlenecks.

Vector Data Layout and Sliding Windows

For the horizontal filtering pass, adjacent destination pixels require overlapping sets of consecutive source pixels. For example, calculating four adjacent outputs using an 8-tap filter requires eleven contiguous source samples (\(4 + 8 - 1\)).

ARM NEON handles this spatial overlap using vector extract and permutation instructions rather than performing multiple unaligned memory accesses:

1. Vector Loading: NEON loads 8-bit unsigned source samples into 128-bit vector registers (uint8x16_t) using contiguous loads (vld1q_u8).
2. Sliding Window via Vector Extract: To align adjacent filter windows without re-loading from memory, implementations use the ext (or vextq_u8 intrinsic) instruction. By pairing two consecutive 128-bit registers, ext extracts an 8-byte or 16-byte slice at specific byte offsets (\(0, 1, 2, \dots, 7\)). This constructs the vectors containing the aligned samples for each tap across all parallel lanes.

Arithmetic Execution and Precision Management

Filter coefficients in AV1 are signed values, typically normalized to sum to 128. Source pixels are unsigned 8-bit or 10-bit integers. Multiplying these values produces signed intermediate results that exceed 8 bits of precision.

To manage dynamic range without overflow:

• Widening Multiplication: On base ARMv8-A architectures, NEON utilizes widening multiply-accumulate operations (smull, smlal, smlal2 or vmull_s8, vmlal_s8). The 8-bit unsigned pixel samples and signed filter taps are processed into 16-bit signed accumulators (int16x8_t).
• Dot Product Acceleration: On ARMv8.2-A and later cores supporting the Dot Product extension, four taps can be collapsed using the signed-by-unsigned dot product instructions (sdot / udot). This computes a 4-element dot product of 8-bit integers into a 32-bit destination vector in a single cycle, processing an entire 8-tap filter in just two multiply-accumulate steps per lane.

Vertical Pass and Memory Interleaving

The vertical filtering pass operates across rows rather than columns. Since pixels in the same column across successive rows reside at constant stride offsets in memory, the data is already aligned across parallel vector lanes.

1. Row Accumulation: NEON loads samples from eight consecutive rows into separate registers.
2. Direct Multiply-Accumulate: Each row register is multiplied by its corresponding scalar filter tap using lane-broadcast multiplication (vmlal_lane_s16 or vmlal_n_s16). Because every vector lane corresponds to an identical tap index along the column, no horizontal byte shifts (ext) are required during the vertical pass.

Normalization and Saturated Narrowing

AV1 requires precise intermediate and final rounding shifts to restore values to the valid pixel range:

1. Rounding Shifts: Intermediate sums are scaled down using rounding shift instructions such as vrshrq_n_s16 or vrshrq_n_s32, which automatically add the required rounding offset (\(2^{\text{shift}-1}\)) before truncation.
2. Saturation: The filtered results are converted back from 16-bit intermediate representation to 8-bit (or 10-bit) storage using saturating narrowing operations (vqmovun_s16). This instruction safely clamps values outside the \([0, 255]\) range without requiring branch instructions, preventing visual artifacts caused by integer wrapping.