Configure find_rect Threshold and Scale in FFmpeg

The FFmpeg find_rect filter is a powerful tool used to detect a template object or logo within a video stream. This guide explains how to configure the filter's search threshold to control detection sensitivity and how to adjust its scaling options using mipmaps to locate objects of varying sizes.

Understanding the find_rect Filter Syntax

The find_rect filter compares a grayscale template image against the video frames. The basic syntax for the filter is:

ffmpeg -i input.mp4 -vf "find_rect=object=template.pgm:threshold=0.5:mipmaps=3" -f null -
  • object: The path to the template image (typically in PGM or BMP format).
  • threshold: The strictness of the matching algorithm.
  • mipmaps: The setting that controls scale-space representation for multi-scale searching.

Configuring the Search Threshold

The threshold parameter determines how closely a region in the video frame must match the template image to trigger a successful detection.

  • Value Range: The threshold is defined as a float value, typically ranging from 0.0 to 1.0.
  • High Threshold (e.g., 0.8 to 0.95): Requires a near-exact match. Use high values to prevent false positives when your template has very distinct, high-contrast features.
  • Low Threshold (e.g., 0.3 to 0.5): Allows for looser matches. Lower values are useful if the video suffers from compression artifacts, noise, or minor color variations, though they increase the risk of false positives.

Example: Setting a Strict Threshold

ffmpeg -i input.mp4 -vf "find_rect=object=logo.pgm:threshold=0.85" output.mp4

Configuring Scaling Options

The find_rect filter does not support arbitrary, continuous scaling configuration. Instead, it utilizes a scale-space pyramid approach controlled by the mipmaps parameter.

  • mipmaps: This integer parameter specifies the number of downscaled levels (pyramid layers) to generate for both the input frame and the template object.
  • How it works: A value of 1 searches only at the original resolution. Higher values (e.g., 3 or 4) create downscaled versions of the image, allowing the filter to detect the template even if it appears smaller in the video than the source template file.
  • Performance Impact: Increasing the mipmaps value increases the search space, which can decrease processing speed.

Example: Searching with 4 Levels of Scaling

ffmpeg -i input.mp4 -vf "find_rect=object=logo.pgm:threshold=0.6:mipmaps=4" output.mp4

Combining Search Threshold, Scaling, and Search Regions

To optimize performance and accuracy, you can restrict the search to a specific region of the screen using coordinate boundaries (xmin, ymin, xmax, ymax). This prevents the filter from wasting CPU cycles searching areas where the object will never appear.

Example: Optimized Search Command

ffmpeg -i input.mp4 -vf "find_rect=object=logo.pgm:threshold=0.7:mipmaps=3:xmin=100:ymin=100:xmax=500:ymax=500" output.mp4

In this optimized command, FFmpeg searches for the template logo.pgm within a 400x400 pixel box in the top-left quadrant of the video, using a moderate match threshold of 0.7 and generating 3 mipmap levels to accommodate size variations.