How to Multiply Pixel Values of Two Videos in FFmpeg

Multiplying the pixel values of two videos in FFmpeg is a powerful technique used for masking, transitions, and creating complex overlay effects. This article provides a quick, step-by-step guide on how to achieve this using the FFmpeg blend filter set to multiply mode. You will learn the exact command-line syntax required, how the pixel multiplication math works, and how to handle videos of different sizes or frame rates.

The Basic Command

To multiply the pixel values of two videos, you use FFmpeg’s blend filter with the all_mode=multiply option. This multiplies the color values of each corresponding pixel from both videos on a scale of 0 to 1 (where 0 is black and 1 is white/maximum value).

Run the following command in your terminal:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[0:v][1:v]blend=all_mode=multiply[out]" -map "[out]" output.mp4

Command Breakdown

-i video1.mp4 -i video2.mp4: Specifies the two input video files.
-filter_complex: Activates the complex filtergraph, which is required when processing multiple input streams.
[0:v][1:v]: Directs the video stream of the first input (index 0) and the second input (index 1) into the blend filter.
blend=all_mode=multiply: Tells FFmpeg to apply the multiply blending mode to all color channels (Y, U, and V, or R, G, and B).
[out]: Labels the resulting blended video stream.
-map "[out]": Maps the labeled output stream to the final output file (output.mp4).

Handling Mismatched Video Resolutions and Frame Rates

The blend filter requires both input videos to have the exact same resolution, pixel format, and frame rate. If your videos do not match, the command will fail or produce unexpected results.

To resolve this, you can scale the second video to match the first video using the scale2ref filter before blending them:

ffmpeg -i video1.mp4 -i video2.mp4 -filter_complex "[1:v][0:v]scale2ref[v2][v1];[v1][v2]blend=all_mode=multiply[out]" -map "[out]" output.mp4

In this command: 1. [1:v][0:v]scale2ref[v2][v1] takes the second video and scales it to match the dimensions of the first video. 2. The scaled streams ([v1] and [v2]) are then fed directly into the blend filter.