Animated Bounding Box in FFmpeg with sendcmd

This article explains how to animate a bounding box over a video using FFmpeg by combining the drawbox and sendcmd filters. By reading from a custom coordinate list in an external text file, you can dynamically update the position and size of a bounding box at specific timestamps, making it ideal for visualizing object detection tracking data.

Step 1: Create the Coordinate Command File

The sendcmd filter reads commands from a text file and dispatches them to other filters at precise timestamps. First, create a text file named commands.txt containing your custom coordinate list.

The format for each line in the file is: timestamp [filter_instance_identifier] [parameter] [value]

Here is an example commands.txt that updates the bounding box's x, y, width (w), and height (h) at specific intervals:

0.0 box x 100, box y 150, box w 50, box h 50;
1.0 box x 120, box y 155, box w 52, box h 52;
2.0 box x 150, box y 170, box w 55, box h 55;
3.5 box x 200, box y 180, box w 60, box h 60;
4.0 box x 250, box y 200, box w 70, box h 70;

Note: Multiple commands for the same timestamp must be separated by commas, and each timestamp block must end with a semicolon.

Step 2: Run the FFmpeg Command

To apply these coordinates to your video, run the following FFmpeg command in your terminal:

ffmpeg -i input.mp4 -filter_complex "sendcmd=f=commands.txt,drawbox=x=0:y=0:w=0:h=0:color=red:t=3@box" -c:a copy output.mp4

How It Works

  • sendcmd=f=commands.txt: This filter loads your custom coordinate list from commands.txt and schedules the parameter changes.
  • drawbox=...: This filter draws the rectangle on the video.
  • @box: This is a custom identifier appended to the drawbox filter. It matches the box target specified in the commands.txt file, ensuring the command signals are routed to this specific filter instance.
  • x=0:y=0:w=0:h=0: These are the default starting coordinates. They will be immediately overridden by the first timestamp entry in your command file.
  • color=red:t=3: Sets the bounding box border color to red and the thickness (t) to 3 pixels.

Achieving Smooth Animation

Because sendcmd applies discrete updates at specific timestamps, the bounding box will jump instantly from one coordinate set to the next. To achieve a smooth tracking animation, write a script (e.g., in Python) to generate a line in commands.txt for every single frame of your video (e.g., at intervals of 0.04 seconds for a 25 fps video).