Using the FFmpeg Threshold Filter with Video Maps

The threshold filter in FFmpeg allows you to perform pixel-by-pixel comparisons between a source video and a threshold map, routing pixels from two alternate sources depending on whether the source pixel value is above or below the threshold. This article explains how the filter functions, details the required four-stream input structure, and provides practical FFmpeg command examples to generate custom video masks and conditional overlays.

Understanding the Threshold Filter Inputs

The threshold filter processes four distinct video streams at the exact same resolution and frame rate. It evaluates each pixel coordinates \((x, y)\) across all streams simultaneously:

  1. Input 1 (Base Stream): The primary video being evaluated.
  2. Input 2 (Threshold Stream): The threshold map. Each pixel's luminance or color value serves as the cutoff limit.
  3. Input 3 (Min Stream): The source used for the output pixel if Base < Threshold.
  4. Input 4 (Max Stream): The source used for the output pixel if Base >= Threshold.

If a pixel in the base video has a value lower than the corresponding pixel in the threshold map, the filter outputs the pixel from the third stream. Otherwise, it outputs the pixel from the fourth stream.

Basic Syntax and Parameters

The basic filter syntax within -filter_complex connects four labeled streams into threshold:

[base][threshold_map][min_source][max_source]threshold=planes=15
  • planes: Specifies which planes (color channels) to process. The value is a bitmask:
    • 1: Y (Luma)
    • 7: Y, U, V (all standard color planes)
    • 15: All planes including Alpha (default).

Generating a Binary Mask Against a Static Map

To compare a video against a static grayscale image mask and output a pure black-and-white binary mask:

ffmpeg -i input.mp4 -loop 1 -i map.png \
-f lavfi -i color=c=black:s=1920x1080:r=30 \
-f lavfi -i color=c=white:s=1920x1080:r=30 \
-filter_complex "[0:v][1:v][2:v][3:v]threshold=planes=7[outv]" \
-map "[outv]" -shortest output.mp4

In this command:

  • [0:v] is the base video.
  • [1:v] is looped from map.png as the threshold map.
  • [2:v] is generated black video (min).
  • [3:v] is generated white video (max).
  • Any pixel in input.mp4 darker than map.png turns black; any pixel equal or brighter turns white.

Conditional Pixel Replacement with Dynamic Maps

You can also use a moving video as the threshold map while preserving original footage on condition. The following example retains original pixels where the video is brighter than a generated gradient, otherwise displaying black:

ffmpeg -i input.mp4 -i threshold_video.mp4 \
-f lavfi -i color=c=black:s=1920x1080:r=30 \
-filter_complex "[0:v][1:v][2:v][0:v]threshold=planes=7[outv]" \
-map "[outv]" output.mp4

Here, [0:v] serves as both the evaluated base and the fourth input stream (max). If input.mp4 meets or exceeds threshold_video.mp4, the original pixel passes through; otherwise, it is replaced by the solid black pixel from the third stream.