How to Use FFmpeg lut2 Filter for Pixel Comparison

This article explains how to use the FFmpeg lut2 filter to perform pixel-by-pixel comparisons between two video streams. You will learn the basic syntax of the filter, how to construct evaluation expressions for different color channels, and see practical examples of highlighting differences between two videos.

Understanding the lut2 Filter

The lut2 (Lookup Table 2) filter in FFmpeg takes two input video streams and outputs a single stream. It computes each pixel of the output by evaluating an expression that compares the corresponding pixels of the two inputs.

This filter is highly efficient because it pre-calculates a 2D lookup table for all possible combinations of 8-bit or 16-bit input pixel values before processing the video frames.

Syntax and Variables

The basic syntax for the lut2 filter is:

lut2=c0='expr':c1='expr':c2='expr':c3='expr'

• c0 refers to the first channel (Y in YUV, R in RGB).
• c1 refers to the second channel (U in YUV, G in RGB).
• c2 refers to the third channel (V in YUV, B in RGB).
• c3 refers to the fourth channel (Alpha, if present).

Within the expressions, you use the following variables to represent the pixel values: * x: The pixel value from the first input video. * y: The pixel value from the second input video.

Practical Examples

1. Visualizing Absolute Difference

To see the exact differences between two videos, you can calculate the absolute difference between their pixel values. If the pixels are identical, the output will be black (0). Any difference will appear as a lighter pixel.

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2=c0='abs(x-y)':c1='abs(x-y)':c2='abs(x-y)'" output.mp4

In this command: * abs(x-y) subtracts the pixel value of the second video from the first and takes the absolute value, preventing negative results. * This expression is applied to all three color channels.

2. Thresholding Differences (Binary Mask)

If you want to highlight only significant differences while ignoring compression artifacts or minor noise, you can use a threshold.

The following command sets pixels to white (255) if the difference is greater than 15, and black (0) if they are nearly identical:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2=c0='gt(abs(x-y),15)*255':c1='128':c2='128'" output.mp4

• gt(abs(x-y),15) returns 1 if the absolute difference is greater than 15, and 0 otherwise.
• Multiplying by 255 scales the result to maximum brightness.
• c1='128' and c2='128' set the U and V chroma channels to neutral grey, resulting in a clean grayscale mask.

3. Highlighting Differences in Color (YUV Space)

If you are working with YUV video and want to highlight differences in brightness (Y) while preserving the original chroma or setting a specific color, you can target only the luma channel (c0):

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2=c0='if(lt(abs(x-y),10),0,255)':c1='128':c2='128'" output.mp4

• if(lt(abs(x-y),10),0,255) checks if the difference is less than 10. If true, it outputs 0 (black). If false, it outputs 255 (white).

Performance Note

Because lut2 builds a lookup table in memory at the start of the process, it is incredibly fast during execution. However, if you are working with high-bitdepth video (e.g., 10-bit or 16-bit), the lookup table requires significantly more memory to initialize.