How to Use the FFmpeg tlut2 Filter

The tlut2 (Temporal Look-Up Table 2) filter in FFmpeg applies custom mathematical expressions across consecutive frames within a single video stream to create temporal visual effects, generate motion-detection masks, or calculate frame-to-frame differences. This guide explains how the filter functions, details its syntax and primary variables, and provides practical, copy-pasteable command examples for common video processing tasks.

Understanding the tlut2 Filter

Unlike standard 2D LUT filters that operate on spatial data or static color maps, tlut2 processes pixels across the time domain. It evaluates the pixel value of the current frame alongside the corresponding pixel value from the immediately preceding frame.

In any given expression:

  • x: Represents the pixel value of the current frame.
  • y: Represents the pixel value of the previous frame.

The filter assigns the calculated result to the current frame's output.

Basic Syntax and Parameters

The basic filter syntax is:

ffmpeg -i input.mp4 -vf "tlut2=c0='expression':c1='expression':c2='expression':c3='expression'" output.mp4
  • c0: Expression for the first component (Y in YUV, R in RGB).
  • c1: Expression for the second component (U in YUV, G in RGB).
  • c2: Expression for the third component (V in YUV, B in RGB).
  • c3: Expression for the alpha component.

If an expression for a component is omitted, it defaults to retaining the current frame's value (x).

Practical Examples

1. Frame Differencing (Temporal Difference)

To visualize movement between frames, compute the absolute difference between the current pixel and the previous pixel.

ffmpeg -i input.mp4 -vf "format=yuv420p,tlut2=c0='abs(x-y)':c1='128':c2='128'" -c:a copy difference.mp4
  • c0='abs(x-y)' calculates the difference in the luma (brightness) plane. Areas with no movement become black (0), while moving objects appear brighter.
  • c1='128' and c2='128' neutralize chroma to render the result as grayscale.

2. Temporal Smoothing / Simple Two-Frame Blur

To blend consecutive frames together for a slight motion-smoothing or temporal ghosting effect, calculate the average of the current and previous pixel values:

ffmpeg -i input.mp4 -vf "tlut2=c0='(x+y)/2':c1='(x+y)/2':c2='(x+y)/2'" -c:a copy smoothed.mp4

3. Binary Motion Mask

To create a high-contrast motion mask that turns pixels pure white if movement exceeds a certain threshold and pure black otherwise:

ffmpeg -i input.mp4 -vf "format=yuv420p,tlut2=c0='if(gt(abs(x-y),25),235,16)':c1='128':c2='128'" -c:a copy motion_mask.mp4
  • gt(abs(x-y), 25) checks if the absolute difference in luminance exceeds the threshold value of 25.
  • If true, it assigns 235 (broadcast white); otherwise, it assigns 16 (broadcast black).

4. Highlighting Temporal Additions

To highlight only newly introduced elements (where current brightness exceeds previous brightness) while ignoring areas where brightness decreased:

ffmpeg -i input.mp4 -vf "format=yuv420p,tlut2=c0='max(x-y,0)':c1='128':c2='128'" -c:a copy additions.mp4

Key Operational Notes

  • Initial Frame Handling: On the very first frame of a stream, there is no preceding frame. FFmpeg uses the first frame's own values for y, resulting in x == y for frame 0.
  • Pixel Ranges: For standard 8-bit video, pixel values typically range from 0 to 255 (or 16 to 235 in limited range YUV). For 10-bit or higher bit-depth video, scale expression constants accordingly (e.g., up to 1023 for 10-bit).
  • Color Space Consistency: Preceding the filter with format=yuv420p or format=rgb24 ensures predictable component mapping across diverse input sources.