How to Use FFmpeg Maskedmerge Filter with Custom Masks

This article explains how to use the FFmpeg maskedmerge filter to blend two video sources using a custom mask. You will learn the core concepts of mask-based merging, how to apply a static image as a custom mask, and how to programmatically generate dynamic, animated masks directly inside FFmpeg using video filters.

Understanding the Maskedmerge Filter

The maskedmerge filter combines three input streams: 1. First Input (Base): The background video. 2. Second Input (Overlay): The foreground video. 3. Third Input (Mask): A grayscale video or image.

The filter processes pixels based on the mask’s luminance value (0-255). Black areas (0) in the mask display the base video, white areas (255) display the overlay video, and gray areas create a semi-transparent blend. All three inputs must have identical dimensions and frame rates.

Method 1: Using a Static Image as a Custom Mask

If you have a pre-designed grayscale PNG image to use as your mask, you can load it as the third input.

Use the following command to merge base.mp4 and overlay.mp4 using mask.png:

ffmpeg -i base.mp4 -i overlay.mp4 -loop 1 -i mask.png -filter_complex "[0:v][1:v][2:v]maskedmerge[out]" -map "[out]" -shortest output.mp4

Command Breakdown:

• -loop 1 -i mask.png: Loops the single-frame mask image so it matches the duration of the videos.
• [0:v][1:v][2:v]: Passes the base video, overlay video, and mask image to the filter in that exact order.
• maskedmerge: Merges the streams.
• -shortest: Tells FFmpeg to stop encoding when the shortest video input ends.

Method 2: Generating a Custom Mask Programmatically

You can create custom shapes and gradients directly inside FFmpeg without external images by using the geq (generic equation) filter to generate a mask stream.

Creating a Static Split-Screen Mask

To split the screen vertically down the middle (left side base, right side overlay), generate a mask that is black on the left and white on the right:

ffmpeg -i base.mp4 -i overlay.mp4 -filter_complex "[0:v]geq=lum='if(gt(X,W/2),255,0)'[mask];[0:v][1:v][mask]maskedmerge[out]" -map "[out]" output.mp4

• geq=lum='if(gt(X,W/2),255,0)': Evaluates every pixel. If the X-coordinate (X) is greater than half the width (W/2), it colors the pixel white (255); otherwise, it colors it black (0).

Creating a Custom Circular Mask

To display the overlay video inside a circle in the center of the screen, use the algebraic equation for a circle:

ffmpeg -i base.mp4 -i overlay.mp4 -filter_complex "[0:v]geq=lum='if(lt(sqrt(pow(X-W/2,2)+pow(Y-H/2,2)),200),255,0)'[mask];[0:v][1:v][mask]maskedmerge[out]" -map "[out]" output.mp4

• This equation calculates the distance of each pixel (X,Y) from the center (W/2, H/2). If the distance is less than 200 pixels, it outputs white, creating a circular mask with a radius of 200 pixels.

Method 3: Creating an Animated Mask Transition

You can introduce the time variable (T) to create dynamic transitions, such as a horizontal wipe from left to right over 5 seconds:

ffmpeg -i base.mp4 -i overlay.mp4 -filter_complex "[0:v]geq=lum='if(gt(X,W*T/5),0,255)'[mask];[0:v][1:v][mask]maskedmerge[out]" -map "[out]" output.mp4

• W*T/5: As time (T) progresses from 0 to 5 seconds, the transition boundary moves from the left edge (0) to the right edge (W).