Combine Two Videos with FFmpeg Maskedmerge Filter

This guide explains how to use the FFmpeg maskedmerge filter to blend two separate video streams using a third grayscale video as an alpha mask. You will learn the exact command-line syntax, how the filter interprets the input streams, and how to handle videos of different dimensions to ensure a clean composite output.

The Basic Command Syntax

The maskedmerge filter requires exactly three video inputs in a specific order: 1. First Input (Base): The background video. 2. Second Input (Overlay): The foreground video. 3. Third Input (Mask): The grayscale video. White pixels (high value) display the second input, black pixels (low value) display the first input, and gray pixels create a semi-transparent blend.

Here is the fundamental FFmpeg command:

ffmpeg -i background.mp4 -i foreground.mp4 -i mask.mp4 -filter_complex "[0:v][1:v][2:v]maskedmerge[out]" -map "[out]" output.mp4

Command Breakdown

-i background.mp4: Maps to input index 0 ([0:v]).
-i foreground.mp4: Maps to input index 1 ([1:v]).
-i mask.mp4: Maps to input index 2 ([2:v]).
[0:v][1:v][2:v]maskedmerge[out]: Takes the three video streams, applies the maskedmerge filter, and outputs the result to a temporary stream named [out].
-map "[out]": Directs FFmpeg to write the processed stream to the final output file.

Handling Different Resolutions and Frame Rates

The maskedmerge filter requires all three input videos to have the exact same resolution and frame rate. If your files do not match, you must resize and sync them using the scale and fps filters before merging them.

Here is an example that forces all inputs to 1920x1080 resolution and 30 frames per second:

ffmpeg -i background.mp4 -i foreground.mp4 -i mask.mp4 -filter_complex \
"[0:v]scale=1920:1080,fps=30[bg]; \
 [1:v]scale=1920:1080,fps=30[fg]; \
 [2:v]scale=1920:1080,fps=30[mask]; \
 [bg][fg][mask]maskedmerge[out]" \
-map "[out]" output.mp4

Hardware Acceleration Options

For high-resolution videos, processing via CPU can be slow. If you have an NVIDIA GPU, you can utilize hardware-accelerated scaling and decoding to speed up the process:

ffmpeg -hwaccel cuda -i background.mp4 -hwaccel cuda -i foreground.mp4 -hwaccel cuda -i mask.mp4 -filter_complex \
"[0:v]scale_cuda=1920:1080[bg]; \
 [1:v]scale_cuda=1920:1080[fg]; \
 [2:v]scale_cuda=1920:1080[mask]; \
 [bg][fg][mask]maskedmerge[out]" \
-c:v h264_nvenc -map "[out]" output.mp4