FFmpeg Concat Filter with Different Codecs

Merging video files with different codecs, resolutions, or frame rates in FFmpeg requires the concat video filter, as the basic demuxer method only works for files with identical properties. This guide explains how to use the FFmpeg concat filter to decode, normalize, and merge mismatched files into a single, seamless video, complete with step-by-step commands and parameter explanations.

Why the Concat Filter is Required

FFmpeg offers two ways to join files: the concat demuxer and the concat filter.

• The demuxer is fast because it does not re-encode, but it fails if your input videos use different codecs (e.g., merging an H.264 MP4 with a VP9 WebM) or have different dimensions.
• The concat filter decodes all input files, processes them through a shared grid, and re-encodes them into a single target codec. This is the only reliable method for merging mismatched files.

The Basic Concat Filter Command

To merge two files with different codecs but the same resolution and audio channel layout, use the following syntax:

ffmpeg -i input1.mp4 -i input2.mkv -filter_complex "[0:v][0:a][1:v][1:a]concat=n=2:v=1:a=1[outv][outa]" -map "[outv]" -map "[outa]" output.mp4

Command Breakdown:

• -i input1.mp4 -i input2.mkv: Imports the two source files. You can add more inputs as needed.
• -filter_complex: Tells FFmpeg to use a complex filtergraph, which is required when handling multiple inputs and outputs.
• [0:v][0:a][1:v][1:a]: Specifies the input streams to feed into the filter.
  • [0:v] is the video stream of the first input (index 0).
  • [0:a] is the audio stream of the first input.
  • [1:v] and [1:a] represent the video and audio of the second input.
• concat=n=2:v=1:a=1: Calls the concat filter.
  • n=2: The total number of input segments to join.
  • v=1: The number of output video streams (always 1).
  • a=1: The number of output audio streams (always 1).
• [outv][outa]: The names assigned to the resulting merged video and audio streams.
• -map "[outv]" -map "[outa]": Maps the filtered outputs to the final output file.

Handling Different Resolutions and Aspect Ratios

If your input videos have different dimensions (e.g., one is 1080p and the other is 720p), the filter graph will fail unless you normalize their resolutions first. You can scale the videos within the same command before passing them to the concat filter.

This command scales both inputs to 1920x1080 and sets the Sample Aspect Ratio (SAR) to 1:1 to prevent stretching:

ffmpeg -i input1.mp4 -i input2.mkv -filter_complex \
"[0:v]scale=1920:1080,setsar=1[v0]; \
 [1:v]scale=1920:1080,setsar=1[v1]; \
 [v0][0:a][v1][1:a]concat=n=2:v=1:a=1[outv][outa]" \
-map "[outv]" -map "[outa]" -c:v libx264 -c:a aac output.mp4

Explanation of the Scaling Chain:

1. [0:v]scale=1920:1080,setsar=1[v0] resizes the first video and labels the temporary output stream as [v0].
2. [1:v]scale=1920:1080,setsar=1[v1] resizes the second video and labels it [v1].
3. [v0][0:a][v1][1:a]concat... uses the newly scaled video streams ([v0] and [v1]) instead of the raw inputs.
4. -c:v libx264 -c:a aac explicitly sets the output video codec to H.264 and the audio codec to AAC for maximum compatibility.

Merging More Than Two Files

To merge three or more files, list all inputs, update the concat input streams, and change the n parameter to match the total number of files.

ffmpeg -i input1.mp4 -i input2.webm -i input3.mov -filter_complex \
"[0:v][0:a][1:v][1:a][2:v][2:a]concat=n=3:v=1:a=1[outv][outa]" \
-map "[outv]" -map "[outa]" output.mp4