How to Use the FFmpeg tlut2 Filter

This guide explains how to use the tlut2 filter in FFmpeg to merge and manipulate two video streams using pixel-by-pixel look-up table expressions. You will learn the basic syntax of the filter, the parameters and variables it supports, and practical command-line examples to help you combine video frames based on their presentation timestamps (PTS).

What is the tlut2 Filter?

The tlut2 (Time-matching Lookup Table 2) filter takes two input video streams, matches their frames according to their timestamps, and outputs a single video stream. It applies a user-defined mathematical expression to the pixel values of both inputs to calculate the output pixel values.

This filter is highly efficient because it pre-calculates the lookup table for all possible pixel combinations (0-255 for 8-bit video) before processing, making it much faster than evaluating expressions per-pixel in real-time.

Basic Syntax and Parameters

The basic structure of the tlut2 filter is:

-filter_complex "[input1][input2]tlut2=c0='expr':c1='expr':c2='expr':c3='expr'[output]"

Key Parameters:

• c0: Expression for the first color plane (Y in YUV, R in RGB).
• c1: Expression for the second color plane (U in YUV, G in RGB).
• c2: Expression for the third color plane (V in YUV, B in RGB).
• c3: Expression for the fourth color plane (Alpha channel, if present).

If an expression for a plane is omitted, it defaults to val, which passes the first input’s pixel value through unchanged.

Available Variables within Expressions:

• val: The pixel value from the first input video.
• val2: The pixel value from the corresponding frame of the second input video.

Practical Examples

1. Adding Pixel Values (Blending Videos)

To blend two videos together by adding their pixel values and capping the result at the maximum 8-bit value (255), use the following command:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]tlut2=c0='clip(val+val2,0,255)':c1='clip(val+val2,0,255)':c2='clip(val+val2,0,255)'[out]" -map "[out]" output.mp4

2. Thresholding / Masking

You can use the second video as a dynamic mask. In this example, if the pixel value in the second video (val2) is greater than 128, the output uses the pixel value from the first video (val). Otherwise, the output pixel is set to 0 (black):

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]tlut2=c0='if(gt(val2,128),val,0)'" output.mp4

3. Creating a Difference Matte

To visualize the absolute difference between two similar video files (useful for quality analysis or motion detection):

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]tlut2=c0='abs(val-val2)':c1='abs(val-val2)':c2='abs(val-val2)'[out]" -map "[out]" difference.mp4