FFmpeg lut2 Filter: Logical Operations Between Two Videos

This guide explains how to use the FFmpeg lut2 filter to perform pixel-by-pixel logical operations between two video streams. You will learn the basic syntax of the filter, how color channels are processed using variables, and see practical command-line examples for operations like bitwise AND, OR, XOR, and threshold masking.

Understanding the lut2 Filter Syntax

The lut2 filter takes two video inputs, compares or combines their pixels using a specified mathematical expression, and outputs a single video.

The basic syntax for the filter is:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]lut2=c0='expression':c1='expression':c2='expression':c3='expression'[out]" -map "[out]" output.mp4

In this syntax: * [0:v] and [1:v] represent the first and second input video streams. * c0, c1, c2, and c3 represent the color channels of the pixel formats (such as Y, U, V, A or R, G, B, A). * expression is the mathematical or logical formula applied to each pixel.

Within the expressions, two key variables are used: * x: The value of the pixel in the first input video (0 to 255 for 8-bit video). * y: The value of the pixel in the corresponding position of the second input video.

Common Logical Operations

FFmpeg’s expression evaluator supports bitwise logical operators that are perfect for pixel manipulation.

Bitwise AND

A bitwise AND operation multiplies the binary representations of the pixel values. This can be used to isolate overlapping elements or create custom mask overlays.

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2=c0='bitand(x,y)':c1='bitand(x,y)':c2='bitand(x,y)'" output.mp4

Bitwise OR

A bitwise OR operation combines the binary values of the pixels, resulting in a brighter, blended image where features from both videos are merged.

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2=c0='bitor(x,y)':c1='bitor(x,y)':c2='bitor(x,y)'" output.mp4

Bitwise XOR (Difference Detection)

A bitwise XOR outputs a value only when the bits of the two input pixels are different. This is highly effective for visual difference detection, motion detection, and transition effects.

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2=c0='bitxor(x,y)':c1='bitxor(x,y)':c2='bitxor(x,y)'" output.mp4

Creating Logical Masks with Thresholds

You can also use ternary operators (if(condition, true_value, false_value)) and comparison operators inside the lut2 filter to create binary masks based on differences.

For example, to create a black-and-white mask showing where the pixels of two videos differ by a absolute value greater than 30:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2=c0='if(gt(abs(x-y),30),255,0)':c1='128':c2='128'" output.mp4

In this command: * abs(x-y) calculates the absolute difference between the first and second video. * gt(..., 30) checks if the difference is greater than 30. * if(..., 255, 0) sets the luma channel (c0) to white (255) if true, and black (0) if false. * c1 and c2 are set to 128 (neutral gray in YUV color space) to ensure the output remains grayscale.