Estimate Motion Vectors with FFmpeg mestimate

This article provides a straightforward guide on how to use the mestimate filter in FFmpeg to calculate motion vectors between video frames. You will learn the correct command-line syntax, understand the key parameters for adjusting block sizes and search methods, and discover how to visualize these motion vectors directly on your video output.

Understanding the mestimate Filter

The mestimate (motion estimation) filter uses block-matching algorithms to look at pixel groups across consecutive frames and estimate their movement. Instead of altering the visual pixels directly, mestimate calculates these movement coordinates and attaches them to the video frames as side data (metadata).

Basic Syntax and Visualization

Because mestimate only calculates and attaches motion data in the background, you will not see any change in the output video by running it alone. To actually see the vectors, you must pair mestimate with the codecview filter, which draws the vectors as overlay arrows.

Use the following command to estimate motion vectors and visualize them on your video:

ffmpeg -i input.mp4 -vf "mestimate,codecview=mv=pf" output.mp4

In this command: * mestimate calculates the motion vectors. * codecview=mv=pf tells FFmpeg to draw the motion vectors using forward predicted (PF) motion spatial coordinates.

Customizing mestimate Parameters

You can customize how the motion search is performed by passing specific options to the mestimate filter. The syntax is mestimate=option=value:option2=value2.

The three primary parameters are:

  1. method: Specifies the motion estimation search algorithm.

    • ds: Diamond search (default, fast).
    • esa: Exhaustive search (slowest, but most accurate).
    • fss: Four-step search.
    • tssa: Three-step search.
    • hexbs: Hexagon-based search.
    • umh: Uneven multi-hexagon search.

  2. mb_size: Sets the macroblock size for block matching. The default is 16 (16x16 pixels). Lowering this (e.g., to 8) increases detail but requires more processing power.

  3. search_param: Sets the search search radius (the window size around the block). The default is 7.

Example with Custom Parameters

If you need highly accurate motion tracking for scientific or detailed editing purposes, you can use the exhaustive search method (esa) with a smaller macroblock size of 8 and a wider search radius of 10:

ffmpeg -i input.mp4 -vf "mestimate=method=esa:mb_size=8:search_param=10,codecview=mv=pf" output.mp4