FFmpeg: Combine Video and Audio Visualizer Side-by-Side

This article provides a straightforward guide on how to use FFmpeg to combine a video stream and a dynamically generated audio visualizer side-by-side into a single output file. You will learn the exact command-line structure, how the filter graph manages the assets, and how to customize the visualizer to match your video’s dimensions.

The FFmpeg Command

To merge a video and an audio visualizer side-by-side, you need to scale both video inputs to the same height and use the hstack (horizontal stack) filter.

Run the following command in your terminal:

ffmpeg -i input.mp4 -filter_complex \
"[0:v]scale=-1:480[vid]; \
 [0:a]showwaves=size=640x480:mode=line:colors=cyan[vis]; \
 [vid][vis]hstack=inputs=2[v]" \
-map "[v]" -map 0:a -c:v libx264 -crf 23 -c:a aac -b:a 192k output.mp4

Command Breakdown

Here is how the command works step-by-step:

• -i input.mp4: Specifies your source file, which contains both the video and audio tracks.
• -filter_complex: Tells FFmpeg to use a complex filter graph because we are dealing with multiple inputs and outputs.
  • [0:v]scale=-1:480[vid]: Takes the video stream ([0:v]), scales its height to 480 pixels while maintaining the aspect ratio (-1), and labels the output as [vid].
  • [0:a]showwaves=...[vis]: Takes the audio stream ([0:a]) and processes it through the showwaves filter to generate a visualizer.
    • size=640x480 matches the height of the scaled video (480px).
    • mode=line draws the waveform as lines.
    • colors=cyan sets the color of the wave.
    • The output stream is labeled as [vis].
  • [vid][vis]hstack=inputs=2[v]: Takes the scaled video [vid] and the visualizer [vis], stacks them side-by-side horizontally, and labels the combined video output as [v].
• -map "[v]": Maps the combined side-by-side video stream to the final output file.
• -map 0:a: Maps the original audio from the input file to the final output file so you can hear the sound.
• -c:v libx264 -c:a aac: Encodes the output video using the H.264 codec and the audio using the AAC codec.

Customizing the Visualizer

You can replace the showwaves filter with other FFmpeg audio visualization filters depending on your preference:

• showvolume: Displays volume meters.

  [0:a]showvolume=w=640:h=480[vis]

• showfreqs: Displays a frequency spectrum.

  [0:a]showfreqs=size=640x480:mode=bar[vis]

• avectorscope: Displays an audio vector scope (great for stereo audio).

  [0:a]avectorscope=size=640x480[vis]

Ensure that whichever filter you choose, the height parameter matches the height of your scaled video stream (e.g., 480) to prevent errors with the hstack filter.