How to Create a Vocoder Effect with FFmpeg

Applying a classic vocoder effect requires merging the tonal characteristics of a carrier signal—such as a rich synthesizer pad—with the frequency envelope of a modulator signal, typically a speech or vocal recording. While FFmpeg does not include a standalone channel-vocoder filter by default, you can achieve authentic robotic cross-synthesis natively using FFmpeg's audio finite impulse response filter (afir) or by bridging standard LADSPA vocoder plugins. This guide explains how to set up your carrier and modulator audio tracks and run the exact commands required to produce the effect.

Understanding Carrier and Modulator Tracks

A classic vocoder relies on two distinct audio inputs:

  • The Carrier (Synth): This provides the pitch and tonal body. For best results, use a bright synthesizer sound rich in harmonics, such as a detuned sawtooth wave or a wide analog pad.
  • The Modulator (Vocal): This provides the rhythmic articulation and formant shape. The vocal should be dry (no reverb or delay), heavily compressed, and clearly spoken or sung.

Method 1: Native Convolution Vocoding Using afir

FFmpeg includes the afir filter, which convolves one audio stream with another. Mathematically, convolving a harmonically rich synth with a vocal track imprints the spectral characteristics and vowels of the voice onto the synth tones.

Basic Command

ffmpeg -i synth.wav -i vocal.wav -filter_complex "[0:a][1:a]afir=dry=0:wet=1:gtype=none[out]" -map "[out]" vocoded_output.wav

In this command:

  • -i synth.wav is input 0 (the carrier signal).
  • -i vocal.wav is input 1 (the modulator impulse).
  • dry=0:wet=1 ensures that only the convolved sound is output.
  • gtype=none prevents unintended automatic gain compensation.

Enhancing Vocal Intelligibility

Unprocessed vocals can sound muffled after convolution. Pre-filtering the vocal track with high-pass and high-shelf filters boosts the consonants (\(S\), \(T\), and \(K\) sounds), drastically improving clarity:

ffmpeg -i synth.wav -i vocal.wav -filter_complex \
"[1:a]highpass=f=150,treble=g=8:f=4000,compand=attacks=0.01:decays=0.1:points=-60/-60|-20/-6|0/0[mod]; \
 [0:a][mod]afir=dry=0:wet=1:gtype=gn[out]" \
-map "[out]" vocoded_output.wav

Method 2: Traditional Filterbank Vocoding via LADSPA

If you require a traditional analog-style filterbank vocoder rather than spectral convolution, use FFmpeg's ladspa filter to load an external vocoder plugin, such as the open-source TAP Vocoder (tap_vocoder.so) or the LADSPA Vocoder (vocoder.so).

Command Syntax

Ensure your system has the LADSPA plugin installed, then specify the carrier and modulator channels into the plugin filter:

ffmpeg -i synth.wav -i vocal.wav -filter_complex \
"[0:a][1:a]amerge=inputs=2[merged]; \
 [merged]ladspa=file=vocoder:filter=vocoder:controls=16|0.5[out]" \
-map "[out]" vocoded_output.wav
  • amerge=inputs=2 combines the carrier and modulator into a single stereo stream where channel 1 is carrier and channel 2 is modulator.
  • file=vocoder loads the LADSPA library.
  • controls=16|0.5 sets plugin parameters, such as the number of active filter bands and release time.

Production Tips for Better Results

  1. Avoid Sub-Bass on the Carrier: High-energy sub-frequencies overwhelm the convolution matrix. Add a high-pass filter around 100 Hz to the synth before processing.
  2. Match Lengths: Ensure the synth track is sustained throughout the entire vocal duration. Any silent gaps in the carrier will result in silence in the output, even if the voice is active.
  3. Blend with White Noise: Adding a quiet white-noise layer to the carrier synth track provides the high-frequency spectrum required for crisp sibilance and plosives.