Temporal Motion Vector Field in AV1 Inter Prediction

This article explores the Temporal Motion Vector Field (TMVF) mechanism in the AV1 video codec, explaining how it functions within inter prediction. You will learn the fundamental purpose of TMVF, how motion vectors are stored and projected from reference frames, and how this data improves the compression efficiency of modern video streams by accurately estimating motion across time.

Understanding TMVF in AV1

In video compression, inter prediction relies heavily on motion vectors (MVs) to eliminate temporal redundancy between successive frames. While spatial motion vector prediction utilizes the vectors of neighboring blocks within the same frame, the Temporal Motion Vector Field (TMVF)—often referred to in AV1 specification as Motion Field Motion Vector (MFMV)—utilizes motion vectors from previously decoded reference frames.

Because physical motion in video content typically persists across multiple frames, projecting past motion trajectories into the current frame provides highly accurate motion candidates without demanding significant bit overhead.

How the TMVF Mechanism Works

The TMVF process in AV1 operates through several systematic steps:

1. Grid-Based Storage: As reference frames are decoded, AV1 stores their reconstructed motion vectors and reference frame indices in a structured, downsampled grid (typically at an 8x8 luma sample resolution). This grid forms the temporal motion vector buffer.

2. Temporal Motion Vector Projection: AV1 inspects the motion vectors stored in the reference frames and projects them onto the current frame. Unlike older standards that simply looked up the collocated block in a reference picture, AV1 calculates where a moving block from a reference frame will land in the current frame based on its trajectory.

3. Temporal Distance Scaling: When projecting motion vectors, AV1 scales the vectors linearly based on the relative display times (or presentation timestamps) between the reference frame, the target frame, and the frame to which the original motion vector pointed. This ensures that speed and direction are accurately adapted to the current time interval.

4. Overlap and Conflict Handling: Multiple projected vectors may occasionally map to the same 8x8 block in the current frame, or some blocks may receive no projected vectors at all. AV1 uses strict priority rules based on reference frame recency to resolve overlaps and leaves unreferenced blocks blank.

Integration into Inter Prediction Candidate Lists

Once the temporal motion vector field for the current frame is constructed, it integrates directly into the AV1 inter-frame prediction pipeline:

• Motion Vector Candidate Lists: When coding a block, AV1 constructs a list of candidate motion vectors to predict its motion. The TMVF supplies temporal candidates that compete alongside spatial neighbors in lists such as NEAR and NEAREST modes.
• Compound Prediction Support: TMVF works in both single-reference (unipredictive) and dual-reference (compound/bipredictive) modes. The mechanism can project motion vectors associated with different reference lists, allowing the decoder to assemble comprehensive compound candidates.

By projecting physical trajectories across time rather than merely checking static collocated positions, the TMVF mechanism significantly reduces the residual energy and bitrate required to encode motion in AV1.