AV1 Motion Vector Candidate Ranking and Pruning

In the AV1 video codec, inter-frame prediction relies heavily on motion vector (MV) signaling, where coding efficiency depends on accurately predicting motion with minimal bit overhead. To achieve this, AV1 builds a dynamic motion vector candidate list for each block using spatial and temporal neighbors. The codec evaluates these candidates through a structured sorting hierarchy and aggressively prunes redundant or low-probability vectors to produce a compact, ranked candidate list signaled to the decoder.

Spatial and Temporal Candidate Assembly

Before ranking and pruning can occur, AV1 scans predefined neighboring locations to populate an initial candidate pool.

1. Spatial Neighbors: The encoder searches causal spatial blocks surrounding the current coding unit, focusing on adjacent blocks directly to the left, above, above-right, and above-left. It also evaluates extended spatial neighbors if primary blocks are unavailable.
2. Temporal Motion Vector Fields (MFMV): AV1 projects motion vectors from previously decoded reference frames onto a temporal motion field. By mapping these reference trajectories to the current block's position, it gathers collocated temporal motion vectors.

Ranking Methodology

Once candidates are gathered, AV1 sorts them into a priority-driven stack based on geometric proximity and reference frame compatibility:

• Reference Frame Matching: Candidates that directly reference the exact same reference frame (or frame pair in compound prediction) as the current block receive the highest priority. Candidates referencing different frames are scaled or deferred to lower-priority tiers.
• Spatial Proximity: Immediate adjacent neighbors (directly left and above) are given precedence over outer or corner neighbors.
• Candidate Type Classification: Candidates are sorted into categories—primarily NEARESTMV (the primary adjacent vector) and NEARMV (secondary neighbors). Candidates requiring motion vector differences are categorized under NEWMV.

The dynamic stack ranks candidates so that the most statistically probable motion vector occupies the lowest indices (such as index 0), ensuring that entropy coding uses the fewest bits for the most likely choice.

Pruning and Redundancy Elimination

To prevent wasting bits on signaling duplicate data and to conserve decoder processing time, the candidate list undergoes a strict pruning process:

• Duplicate Detection: As each candidate is considered for addition to the final list, its motion vector components \((dx, dy)\) are compared against all candidates already present. If a candidate matches an existing vector, it is immediately discarded.
• Component-Level Thresholding: For specific modes, candidates with imperceptible differences or those that collapse to identical motion trajectories in scaled spaces are merged or omitted.
• List Capacity Truncation: AV1 restricts the final dynamic reference list (DRL) to a fixed maximum length (typically 2 to 4 candidates depending on the prediction mode). Once the maximum capacity of unique, valid vectors is reached, the search terminates, and all remaining uninspected candidates are dropped.

Fallback and Padding

If the pruning process leaves the dynamic candidate list short of the required capacity, AV1 fills the remaining slots using deterministic fallbacks. It injects global motion parameters associated with the current reference frame, or defaults to zero motion vectors \((0, 0)\). This guarantees that the encoder and decoder always maintain identical list lengths without explicit signaling overhead.