FFmpeg Color Space Conversion Before and After LUT

Applying a Look-Up Table (LUT) to a video in FFmpeg requires the input video to match the exact color space and transfer characteristics (gamma) that the LUT was designed for. This article demonstrates how to use FFmpeg’s video filters to convert your video’s color space immediately before applying a 3D LUT, and how to convert it to the desired output format afterward.

Why Convert Color Spaces Around a LUT?

A 3D LUT (typically a .cube file) maps specific input RGB values to specific output RGB values. If a LUT is designed for a flat Log profile (like Sony S-Log3 or Canon Log) or a linear color space, applying it directly to a standard Rec. 709 video will result in heavily distorted colors and contrast. To get accurate results, you must chain your filters so that:

1. Pre-LUT: The input video is converted to the color space expected by the LUT.
2. LUT Application: The LUT is applied via the lut3d filter.
3. Post-LUT: The resulting video is converted to the final target color space (usually Rec. 709 or Rec. 2020) if the LUT output does not already match the target format.

Method 1: Using the zscale Filter (Recommended)

The zscale filter (part of the zimg library) is the most powerful and precise tool in FFmpeg for handling color space, transfer characteristics (gamma), and color primaries. To use it, your FFmpeg build must be compiled with --enable-libzimg.

Example 1: Converting to Linear Space and Back

Some LUTs are designed to operate on linear light. Here is how to convert a standard Rec. 709 video to linear space, apply the LUT, and convert it back to Rec. 709:

ffmpeg -i input.mp4 -vf "zscale=transfer=linear,lut3d=linear_lut.cube,zscale=transfer=bt709" output.mp4

Example 2: Log to Rec. 709 Conversion

If you have a Rec. 2020 HDR video but your LUT expects Rec. 709 (SDR) input, you must downconvert the primaries and transfer characteristics before applying the LUT:

ffmpeg -i input_hdr.mp4 -vf "zscale=primaries=bt709:transfer=bt709:matrix=bt709,lut3d=rec709_lut.cube" output_sdr.mp4

Key zscale Parameters:

• p or primaries: Defines the color primaries (e.g., bt709, bt2020, smpte432 for DCI-P3).
• t or transfer: Defines the gamma curve (e.g., bt709, linear, smpte2084 for HDR10/PQ, arib-std-b67 for HLG).
• m or matrix: Defines the color matrix coefficients (e.g., bt709, bt2020nc).

Method 2: Using the Native colorspace Filter

If your FFmpeg installation does not support zscale, you can use the native colorspace filter. It is slightly less flexible but highly effective for standard broadcast formats.

To convert a BT.2020 video to BT.709 before applying a LUT:

ffmpeg -i input.mp4 -vf "colorspace=all=bt709:trc=bt709,lut3d=my_lut.cube" output.mp4

To convert BT.709 to BT.2020 after applying a LUT:

ffmpeg -i input.mp4 -vf "lut3d=my_lut.cube,colorspace=all=bt2020:trc=smpte2084" output.mp4

Key colorspace Parameters:

• all: Sets primaries, transfer characteristics, and matrix at once (e.g., bt709, bt2020).
• trc: Transfer characteristics (e.g., bt709, smpte2084, smpte170m).

Combining Transformations in a Single Filtergraph

For complex pipelines where you must convert color spaces both before and after the LUT, chain the filters together in a comma-separated list within the -vf flag. FFmpeg processes these filters sequentially from left to right.

ffmpeg -i input_log.mp4 -vf "zscale=p=bt709:t=linear:m=bt709,lut3d=lut_in_linear_space.cube,zscale=p=bt2020:t=smpte2084:m=bt2020nc" output_hdr.mp4

In this command: 1. The S-Log/Rec. 709 input is converted to Linear RGB. 2. The LUT is applied while the video is in Linear RGB. 3. The output of the LUT is converted to HDR10 (Rec. 2020 / PQ) for the final export.