How to Use FFmpeg Mestimate to Visualize Motion Vectors

This article explains how to use the mestimate filter in FFmpeg to estimate and visualize motion vectors in a video. You will learn the exact FFmpeg command-line syntax required to calculate motion estimation and render these vectors directly onto your video output using the companion codecview filter.

Understanding the Filters

To visualize motion vectors of any video (even those without pre-existing motion vector metadata), you must chain two FFmpeg filters together:

1. mestimate (Motion Estimation): This filter analyzes consecutive video frames, calculates the movement of pixel blocks, and exports this motion data as frame metadata.
2. codecview (Codec Viewer): This filter reads the motion vector metadata and draws them on top of the video as colored arrows.

The Basic Command

To calculate and visualize motion vectors using the default settings, run the following command in your terminal:

ffmpeg -i input.mp4 -vf "mestimate,codecview=mv=pf+bf+bb" output.mp4

Here is what each part of the filter chain (-vf) does: * mestimate: Triggers the motion estimation analysis on the input video. * codecview=mv=pf+bf+bb: Draws the motion vectors. * pf displays P-frame (predicted) forward motion vectors (represented as green arrows). * bf displays B-frame (bi-directional predicted) forward motion vectors (represented as blue arrows). * bb displays B-frame backward motion vectors (represented as red arrows).

Customizing the Motion Estimation

You can fine-tune the mestimate filter by adjusting its parameters to change the accuracy and speed of the motion detection.

The basic syntax for customizing mestimate is:

mestimate=method=value:mb_size=value:search_param=value

• method: Specifies the motion estimation search method.
  • esa: Exhaustive search (slowest, most accurate).
  • tss: Three-step search (faster, medium accuracy).
  • fss: Four-step search.
  • ntss: New three-step search.
  • tdls: Two-dimensional logarithmic search.
  • hex: Hexagon-based search.
• mb_size: Sets the macroblock size (default is 16 for 16x16 pixels).
• search_param: Sets the search parameter radius (default is 7).

Example with Custom Settings

If you want to use the highly accurate Exhaustive Search (esa) method with a smaller macroblock size of 8 for more granular vector arrows, use this command:

ffmpeg -i input.mp4 -vf "mestimate=method=esa:mb_size=8,codecview=mv=pf" output.mp4

This command will output a video showing highly detailed forward-predicted motion vectors overlaid on your media.