Using FFmpeg lut2 Filter for Logical Operations

The lut2 filter in FFmpeg allows you to perform pixel-by-pixel comparisons, bitwise calculations, and logical operations between two synchronized video streams using a 2D lookup table. By evaluating mathematical expressions across individual color planes, lut2 enables advanced video blending, masking, and differencing.

Understanding lut2 Syntax and Variables

The lut2 filter accepts two video inputs and processes each color plane independently using expression evaluation.

The basic command structure is:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]lut2=c0='expr':c1='expr':c2='expr'[out]" -map "[out]" output.mp4

Within each expression:

  • x: Represents the pixel value of the first input ([0:v]).
  • y: Represents the pixel value of the second input ([1:v]).
  • c0, c1, c2, c3: Target planes 0 through 3 (Y, U, V, and Alpha in YUV formats; R, G, B, and Alpha in RGB formats).

Bitwise Logical Operations

FFmpeg’s expression evaluator supports native bitwise operations for integer operations on pixel values:

  • Bitwise AND (bitand): Keeps bits that are set in both inputs.
  • Bitwise OR (bitor): Keeps bits that are set in either input.
  • Bitwise XOR (bitxor): Inverts matching bits, highlighting variations.

Example: Bitwise AND

To compute a bitwise AND across all three color channels:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex \
"[0:v][1:v]lut2=c0='bitand(x,y)':c1='bitand(x,y)':c2='bitand(x,y)'" \
-c:a copy output_and.mp4

Example: Bitwise XOR

Bitwise XOR is useful for detecting bit differences between frames:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex \
"[0:v][1:v]lut2=c0='bitxor(x,y)':c1='bitxor(x,y)':c2='bitxor(x,y)'" \
-c:a copy output_xor.mp4

Conditional and Boolean Operations

Logical tests are created using evaluation functions such as if(x, y, z), eq(x, y), gt(x, y), lt(x, y), gte(x, y), and lte(x, y).

Example: Difference Threshold Mask

To generate a high-contrast binary mask showing where two video streams deviate by an absolute difference greater than 20:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex \
"[0:v][1:v]lut2=c0='if(gt(abs(x-y),20),255,0)':c1='128':c2='128'" \
-c:a copy output_diff_mask.mp4

In this YUV example, the luma channel (c0) becomes white (255) when the difference exceeds 20 and black (0) otherwise, while the chroma channels (c1 and c2) are locked to neutral gray (128).

Example: Maximum / Minimum Selection (Logical OR / AND Approximation)

To select only the brighter pixels between two videos:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex \
"[0:v][1:v]lut2=c0='max(x,y)':c1='max(x,y)':c2='max(x,y)'" \
-c:a copy output_max.mp4

Important Requirements

  • Dimensions and Framerate: Both input streams must share identical resolutions and frame rates. Use the scale and fps filters beforehand if the streams do not match.
  • Pixel Formats: Ensure both videos are in the same pixel format (such as yuv420p or rgb24) using format=yuv420p before the lut2 filter to avoid mismatched channel processing.