Export Motion Vectors from FFmpeg mestimate

This article explains how to extract and export the motion vector data generated by the FFmpeg mestimate (motion estimation) filter. By combining the mestimate filter with ffprobe via the lavfi virtual input device, you can calculate motion vectors for any video and export the raw coordinates to a structured JSON file for external analysis.

How the mestimate Filter Works

The mestimate filter estimates motion vectors between consecutive video frames. Instead of modifying the visual stream directly, it appends this motion data to each video frame as side data (specifically, AV_FRAME_DATA_MOTION_VECTORS).

To export this hidden side data into a readable format, you must use ffprobe to read the filter’s output and serialize it into JSON, XML, or CSV.

The Export Command

Run the following command in your terminal to process a video, calculate the motion vectors, and save the output to a JSON file:

ffprobe -f lavfi -i "movie=input.mp4,mestimate" -show_entries frame_side_data_list -of json > motion_vectors.json

Command Breakdown

• -f lavfi: Forces the input format to Libavfilter. This allows ffprobe to run a filtergraph directly.
• -i "movie=input.mp4,mestimate": Defines the input filtergraph. It loads your video file (input.mp4) using the movie source filter and pipes it directly into the mestimate filter.
• -show_entries frame_side_data_list: Instructs ffprobe to ignore default stream information and only print the frame side data list, which contains the motion vectors.
• -of json: Formats the output as a JSON structure.
• > motion_vectors.json: Redirects the standard output to a file named motion_vectors.json.

Understanding the JSON Output

The resulting JSON file will contain an array of frames, each containing a side_data_list. Within this list, you will find blocks of motion vector data structured like this:

{
  "side_data_type": "Motion vectors",
  "motion_vectors": [
    {
      "source": -1,
      "w": 16,
      "h": 16,
      "src_x": 24,
      "src_y": 40,
      "dst_x": 26,
      "dst_y": 39,
      "flags": "0x0"
    }
  ]
}

• w / h: The width and height of the macroblock (typically 16x16, 8x8, etc.).
• src_x / src_y: The starting X and Y coordinate of the block in the source (previous) frame.
• dst_x / dst_y: The destination X and Y coordinate where the block moved in the current frame.
• source: The directional source of the predictor (negative values usually represent past frames, positive values represent future frames in bidirectional B-frames).

Filtering for Faster Exports

If your video is long, the resulting JSON file can become extremely large. You can limit the export to a specific duration using the -to or -t flag:

ffprobe -f lavfi -i "movie=input.mp4,mestimate" -to 00:00:05 -show_entries frame_side_data_list -of json > motion_vectors_5s.json

This command limits the extraction to the first 5 seconds of the video, saving processing time and disk space.