How to Export FFmpeg mestimate Motion Vector Data

This guide explains how to extract and export numerical motion vector data generated by the FFmpeg mestimate filter. While FFmpeg is commonly used to visually draw motion vectors onto video frames, the mestimate filter generates motion vectors dynamically as frame side data (AV_FRAME_DATA_MOTION_VECTORS). By passing this filter graph through ffprobe, you can dump the raw coordinates, block sizes, and displacement values directly into structured formats such as JSON, CSV, or XML.

How mestimate Stores Data

The mestimate (motion estimation) filter calculates displacement between consecutive frames using algorithms such as Diamond Search (ds), Exhaustive Search (esa), or Epzs (epzs).

Instead of writing text directly to the console, the filter attaches this information to each frame as side data. To read and export this side data into a machine-readable format, you must query it using ffprobe via the lavfi (Libavfilter) virtual input device.

Exporting Motion Vector Data with FFprobe

To calculate motion vectors on an existing video and output the raw data to a JSON file, run the following command:

ffprobe -v error -f lavfi -i "movie=input.mp4,mestimate=method=epzs" \
  -show_entries frame=pkt_pts_time:frame_side_data_list \
  -of json > motion_vectors.json

Command Breakdown:

  • -v error: Suppresses non-essential log messages, keeping the output clean.
  • -f lavfi: Uses the Libavfilter virtual input device, allowing filters to be applied before ffprobe reads frame packets.
  • -i "movie=input.mp4,mestimate=method=epzs": Loads the target video file and applies motion estimation. You can change epzs to other methods like esa or ds.
  • -show_entries frame=pkt_pts_time:frame_side_data_list: Instructs ffprobe to output only the frame presentation timestamp and the attached side data (the motion vectors).
  • -of json: Formats the output as structured JSON. You can replace this with csv, xml, or default.
  • > motion_vectors.json: Redirects the stdout stream to a file.

Understanding the Exported JSON Structure

When exported to JSON, each motion vector will appear inside the side_data_list array for each frame:

{
  "side_data_type": "Motion vectors",
  "motion_vectors": [
    {
      "source": -1,
      "w": 16,
      "h": 16,
      "src_x": 48,
      "src_y": 64,
      "dst_x": 52,
      "dst_y": 62,
      "flags": "0x0"
    }
  ]
}

Key Fields:

  • source: The reference frame used (a negative value indicates a past frame, while a positive value indicates a future frame).
  • w and h: The width and height of the macroblock (typically 16x16).
  • src_x and src_y: The center coordinates of the block in the reference frame.
  • dst_x and dst_y: The center coordinates of the block in the current frame.
  • Displacement (\(\Delta X, \Delta Y\)): Calculated by subtracting src_x from dst_x and src_y from dst_y.

Exporting Directly to CSV

If you need tabular data for analysis in tools like Python, R, or Excel, use the CSV printer:

ffprobe -v error -f lavfi -i "movie=input.mp4,mestimate=method=epzs" \
  -show_entries frame_side_data_list \
  -of csv > motion_vectors.csv

This generates a comma-separated list detailing every motion vector found on each block across all processed frames.