AV1 8-Parameter Motion Transformations Explained

This article examines the mathematical transformations used to represent complex camera and object movement in the AV1 video codec, focusing on 8-parameter motion models. It details the underlying geometry of the projective transformation (homography), contrasts it with traditional 6-parameter affine models, and explains the linear algebraic techniques—such as the Direct Linear Transformation and Least Squares estimation—used to derive these transformation matrices from motion vectors.

Projective Transformation vs. Affine Models

In 2D planar kinematics, a standard affine transformation contains 6 degrees of freedom (covering translation, rotation, scaling, and shear):

\[x' = a_{00}x + a_{01}y + a_{02}\] \[y' = a_{10}x + a_{11}y + a_{12}\]

When motion modeling is expanded to 8 parameters in video coding, the underlying mathematical model is formally a projective transformation (also known as a planar homography or perspective transformation). Unlike an affine transformation, which preserves parallelism, a projective transformation models perspective distortion, allowing lines to converge as they recede into the distance.

In homogeneous coordinates, the mapping from original coordinates \((x, y, 1)^T\) to transformed coordinates \((x', y', 1)^T\) is represented by a \(3 \times 3\) matrix:

\[\begin{bmatrix} x' \cdot w \\ y' \cdot w \\ w \end{bmatrix} = \begin{bmatrix} h_{00} & h_{01} & h_{02} \\ h_{10} & h_{11} & h_{12} \\ h_{20} & h_{21} & 1 \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}\]

Expanding this into Cartesian coordinates yields the non-linear rational equations:

\[x' = \frac{h_{00}x + h_{01}y + h_{02}}{h_{20}x + h_{21}y + 1}\]

\[y' = \frac{h_{10}x + h_{11}y + h_{12}}{h_{20}x + h_{21}y + 1}\]

The eight free parameters (\(h_{00}, h_{01}, h_{02}, h_{10}, h_{11}, h_{12}, h_{20}, h_{21}\)) define translation, rotation, anisotropic scaling, shear, and two perspective tilt components (\(h_{20}\) and \(h_{21}\)).

Derivation via Direct Linear Transformation (DLT)

To calculate the 8-parameter matrix from observed frame-to-frame displacements, encoders use the Direct Linear Transformation (DLT) algorithm formulated as a linear system:

1. Multiplying Out the Denominator: For a known correspondence between point \((x_i, y_i)\) in the reference frame and \((x'_i, y'_i)\) in the current frame, the rational equations are rearranged linearly: \[h_{00}x_i + h_{01}y_i + h_{02} - h_{20}x_i x'_i - h_{21}y_i x'_i = x'_i\] \[h_{10}x_i + h_{11}y_i + h_{12} - h_{20}x_i y'_i - h_{21}y_i y'_i = y'_i\]

2. Forming the Linear System: Each point correspondence generates two independent linear equations. Because there are 8 unknowns, a minimum of 4 non-collinear point correspondences are required to construct a solvable linear system of the form: \[\mathbf{A}\mathbf{h} = \mathbf{b}\] where \(\mathbf{A}\) is a \(2N \times 8\) matrix (for \(N \ge 4\) points), \(\mathbf{h}\) is the vector of parameters \([h_{00}, h_{01}, h_{02}, h_{10}, h_{11}, h_{12}, h_{20}, h_{21}]^T\), and \(\mathbf{b}\) is the vector of target coordinates.

Least Squares Estimation and Robust Fitting

In practical AV1 encoder implementations, hundreds of motion vectors are matched between frames. To compute the optimal 8 parameters across noisy data, the overdetermined system is solved using Ordinary Least Squares (OLS):

\[\mathbf{h} = (\mathbf{A}^T \mathbf{A})^{-1} \mathbf{A}^T \mathbf{b}\]

To prevent local object movement or compression artifacts from corrupting the global motion matrix, the derivation pipeline incorporates:

• RANSAC (Random Sample Consensus): Random subsets of 4 point correspondences are evaluated iteratively to estimate candidate matrices and identify inliers.
• Singular Value Decomposition (SVD): Employed when solving the homogeneous equation system \(\mathbf{A}\mathbf{h} = \mathbf{0}\) to find the eigenvector corresponding to the smallest eigenvalue, ensuring numerical stability.

Implementation in AV1 Global and Warped Motion

While the 8-parameter projective transformation serves as the analytical model for perspective and camera calibration during encoder analysis, the final AV1 bitstream specification converts or restricts these models to a 6-parameter affine representation for block-level warped motion compensation. This restriction avoids division operations per pixel during decoding while still benefiting from the higher-order geometric derivation performed during the global motion search.