Combine Two Videos with a Grayscale Mask in FFmpeg

This guide explains how to composite two video streams using a third grayscale video as a matte or transparency map with FFmpeg's maskedmerge filter. You will learn the correct input hierarchy, command-line syntax, and formatting requirements necessary to ensure all three streams merge seamlessly without frame size or pixel format conflicts.

Understanding the maskedmerge Filter

The maskedmerge filter takes exactly three video streams:

  1. First input ([0:v]): The base background video (visible where the mask is black).
  2. Second input ([1:v]): The overlay foreground video (visible where the mask is white).
  3. Third input ([2:v]): The grayscale mask video controlling the transition or transparency between the first two inputs.

Values between 0 (black) and 255 (white) produce semi-transparent blending between the first and second inputs.

Basic Command Syntax

If all three videos already share the exact same resolution, frame rate, and pixel format, use the following command:

ffmpeg -i base.mp4 -i overlay.mp4 -i mask.mp4 -filter_complex "[0:v][1:v][2:v]maskedmerge[v]" -map "[v]" -map 0:a? -c:v libx264 -c:a copy output.mp4

In this command:

  • -i base.mp4, -i overlay.mp4, and -i mask.mp4 specify the base, overlay, and mask respectively.
  • -filter_complex "[0:v][1:v][2:v]maskedmerge[v]" maps the three video streams directly into the filter.
  • -map "[v]" routes the merged video output to the destination file.
  • -map 0:a? includes audio from the base video if present.

Ensuring Matching Dimensions and Pixel Formats

The maskedmerge filter fails if the dimensions or color planes of the inputs do not match. To prevent errors, normalize the resolution and convert the mask to match the primary streams within the filter graph:

ffmpeg -i base.mp4 -i overlay.mp4 -i mask.mp4 -filter_complex "\
[0:v]format=yuv420p[base]; \
[1:v]scale=1920:1080,format=yuv420p[overlay]; \
[2:v]scale=1920:1080,format=yuv420p[mask]; \
[base][overlay][mask]maskedmerge=planes=15[outv]" \
-map "[outv]" -map 0:a? -c:v libx264 -pix_fmt yuv420p output.mp4

Key Parameters Explained

  • scale=1920:1080: Resizes inputs to match the target frame dimensions. Replace with the native resolution of your base footage.
  • format=yuv420p: Standardizes the pixel format across all three streams so the merge operation can process the planes identically.
  • planes=15: Instructs the filter to process the Y (luminance), U, and V (chrominance) planes, along with alpha if present. By default, this applies the luminance values of the mask across all color channels of the blended image.