FFmpeg lut2 Filter Math Expression Guide

The FFmpeg lut2 filter computes a 2-input lookup table that processes two video streams of identical dimensions and pixel formats on a per-pixel basis using mathematical expressions. Unlike the per-frame evaluation of the geq filter, lut2 computes the expression values once across the input range (e.g., 256x256 values for 8-bit video) into a lookup table, resulting in significantly faster processing. This guide explains how to define components, utilize the built-in variables, and construct functional math expressions for video manipulation.

Understanding lut2 Syntax

The filter operates on color planes, which can be defined by individual component parameters (c0, c1, c2, c3):

ffmpeg -i input1.mp4 -i input2.mp4 -filter_complex "[0:v][1:v]lut2=c0='expr':c1='expr':c2='expr':c3='expr'[out]" -map "[out]" output.mp4
  • c0: First color component (Y in YUV, R in RGB).
  • c1: Second color component (U in YUV, G in RGB).
  • c2: Third color component (V in YUV, B in RGB).
  • c3: Alpha channel component (if present).

If all planes use the same formula, the shorthand all='expr' can be used instead of specifying each component individually.

Built-in Variables and Constants

Expressions in lut2 accept standard FFmpeg mathematical functions and variables:

  • x: The value of the pixel from the first input stream.
  • y: The value of the pixel from the second input stream.
  • minval: The minimum allowed pixel value for the component (typically 0).
  • maxval: The maximum allowed pixel value (e.g., 255 for 8-bit, 1023 for 10-bit).
  • clipval(val): A helper function to clamp val between minval and maxval.
  • bitdepth: The component bit depth (e.g., 8, 10, 12).

Practical Math Expression Examples

1. Frame Difference Detection

To calculate the absolute difference between two inputs (useful for motion detection or comparing compression artifacts):

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]lut2=all='abs(x-y)'" output.mp4

2. 50/50 Frame Averaging

To compute the arithmetic mean of two streams:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]lut2=all='(x+y)/2'" output.mp4

3. Multiply Blend Mode

To multiply two video signals normalized by the bit depth:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]lut2=all='clipval((x*y)/maxval)'" output.mp4

4. Conditional Masking

To conditionally replace pixels from the first video based on a threshold from the second video (e.g., if the mask pixel value is greater than half the maximum range):

ffmpeg -i source.mp4 -i mask.mp4 -filter_complex "[0:v][1:v]lut2=all='if(gt(y, maxval/2), x, 0)'" output.mp4

5. Screen Blend Mode

To implement a standard screen blending mode:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]lut2=all='clipval(maxval - ((maxval-x)*(maxval-y))/maxval)'" output.mp4

Tips for Writing Expressions

  1. Always Clamp Out-of-Bounds Results: Expressions that exceed maxval or drop below minval can cause unexpected wrap-around artifacts. Wrap arithmetic that may overflow or underflow inside clipval().
  2. Account for Pixel Format Ranges: When using YUV video, remember that standard dynamic range video often uses limited range (Y: 16–235, UV: 16–240) rather than full range (0–255). Ensure your math reflects the correct range to avoid crushed blacks or clipped highlights.