Custom Math Expressions for FFmpeg lut2 Filter

The FFmpeg lut2 filter is a powerful tool for merging and manipulating two video streams using look-up tables. This article provides a straightforward guide on how to write custom math expressions for the lut2 filter, explaining the available variables, operators, and practical examples to help you achieve advanced pixel-level blending, masking, and color adjustments.

Understanding the lut2 Filter Syntax

The lut2 filter takes two video inputs of the same dimensions and pixel format and outputs a single video. It evaluates a custom math expression for each pixel component (Y, U, V, A or R, G, B, A) to determine the output pixel value.

The basic syntax is:

ffmpeg -i input1.mp4 -i input2.mp4 -filter_complex "lut2=c0='expr':c1='expr':c2='expr':c3='expr'" output.mp4

Where:

  • c0: First plane (Y in YUV, R in RGB)
  • c1: Second plane (U in YUV, G in RGB)
  • c2: Third plane (V in YUV, B in RGB)
  • c3: Fourth plane (Alpha channel)

If you want to apply the same expression to all color planes (excluding alpha), you can use the shorthand:

lut2='expression'

Key Variables

Within your math expressions, you can reference several built-in variables representing the state of the current pixel:

  • x: The value of the pixel at the current coordinate in the first input (input1).
  • y: The value of the pixel at the current coordinate in the second input (input2).
  • w / h: The width and height of the video.
  • bitdepth: The bit depth of the input video (e.g., 8, 10, or 16).
  • maxval: The maximum value possible for a pixel (e.g., 255 for 8-bit, 1023 for 10-bit).

Useful Math Functions

FFmpeg's expression evaluator supports standard operators (+, -, *, /) alongside specific logic functions:

  • min(A, B) / max(A, B): Returns the minimum or maximum of two values.
  • clip(val, min, max): Clamps val between the min and max range.
  • abs(A): Returns the absolute value of A.
  • gt(A, B): Returns 1 if A > B, otherwise 0.
  • lt(A, B): Returns 1 if A < B, otherwise 0.
  • eq(A, B): Returns 1 if A == B, otherwise 0.
  • if(cond, A, B): Returns A if cond is non-zero, otherwise returns B.

Practical Examples

1. Average Blending (50% Opacity Overlay)

To blend two videos equally, add the pixel values together and divide by 2:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2='(x+y)/2'" output.mp4

2. Difference Matte (Comparing Two Videos)

To highlight the visual differences between two video files, subtract the pixel values and use the absolute function to avoid negative numbers:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2='abs(x-y)'" output.mp4

3. Screen Blend Mode

To simulate the classic "Screen" layer style (which brightens the output based on the lightness of the inputs) in 8-bit color space:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2='255-((255-x)*(255-y)/255)'" output.mp4

4. Conditional Masking (Thresholding)

If you want to output the pixel from the first video (x) only if it is brighter than the corresponding pixel in the second video (y), and output black (0) otherwise:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "lut2='if(gt(x,y),x,0)'" output.mp4