How to Use FFmpeg Maskedmerge Filter to Blend Videos

This guide explains how to use the FFmpeg maskedmerge filter to blend two video streams together using a third video as a mask. You will learn the fundamental mechanics of the filter, the required command syntax, and how to ensure your input files are properly formatted for a seamless transition or overlay effect.

Understanding the Maskedmerge Filter

The maskedmerge filter in FFmpeg merges three video streams. It uses the pixel values of a third “mask” video to determine how to blend the first two videos.

The filter requires exactly three video inputs in a specific order: 1. Base Video (Input 0): The background video. It appears where the mask is black (luma value of 0). 2. Overlay Video (Input 1): The foreground video. It appears where the mask is white (luma value of 255). 3. Mask Video (Input 2): A grayscale video that dictates the blend. Gray areas (luma values between 0 and 255) create a semi-transparent, blended transition between the base and overlay videos.

The Basic Command Syntax

To run the maskedmerge filter, you must use FFmpeg’s -filter_complex flag to map the three inputs. Here is the standard command:

ffmpeg -i base.mp4 -i overlay.mp4 -i mask.mp4 -filter_complex "[0:v][1:v][2:v]maskedmerge[out]" -map "[out]" output.mp4

Command Breakdown:

• -i base.mp4: Defines the first input ([0:v]).
• -i overlay.mp4: Defines the second input ([1:v]).
• -i mask.mp4: Defines the third input ([2:v]), which acts as the luma mask.
• [0:v][1:v][2:v]maskedmerge[out]: Feeds the three video streams into the maskedmerge filter in the correct sequence and labels the resulting stream as [out].
• -map "[out]": Directs FFmpeg to write the processed stream to the final output file.

Important Requirements for Best Results

For the maskedmerge filter to work correctly without errors, your input videos must meet the following criteria:

1. Matching Resolutions

All three videos (base, overlay, and mask) must have the exact same width and height. If they do not match, you must scale them within the filtergraph before applying maskedmerge.

Here is an example of scaling the overlay and mask to match a \(1920 \times 1080\) base video:

ffmpeg -i base.mp4 -i overlay.mp4 -i mask.mp4 -filter_complex "[1:v]scale=1920:1080[scaled_overlay]; [2:v]scale=1920:1080[scaled_mask]; [0:v][scaled_overlay][scaled_mask]maskedmerge[out]" -map "[out]" output.mp4

2. Matching Frame Rates and Durations

If your inputs have different frame rates or lengths, the output may stutter or end prematurely. You can append the fps filter to normalize the frame rates, or use the shortest=1 option if you want the output to stop as soon as the shortest video ends.

3. Pixel Format Compatibility

The filter works best when all inputs share the same pixel format (such as yuv420p). FFmpeg generally handles format conversions automatically, but you can force a pixel format using the format filter if you encounter color distortion.