FFmpeg Headphone Filter: Create Surround Sound Mix

This guide explains how to use the FFmpeg headphone filter to convert multi-channel audio, such as 5.1 or 7.1 surround sound, into a spatialized binaural stereo mix for headphones. You will learn how the filter utilizes Head-Related Impulse Response (HRIR) files to simulate a 3D speaker environment, along with the exact FFmpeg commands required to configure the filter and map your audio channels.

Understanding the Headphone Filter

The headphone filter in FFmpeg uses Head-Related Transfer Functions (HRTF) to mimic how human ears perceive sound coming from different directions in a physical space. To achieve this, the filter requires two main components: 1. The Multi-channel Source Audio: Your 5.1, 7.1, or other multi-channel audio track. 2. HRIR (Head-Related Impulse Response) Files: Audio files (usually in WAV format) that contain the acoustic profile of speakers placed at specific angles around a listener’s head.

Step 1: Obtain HRIR Files

To use the filter, you need HRIR coefficients. You can find free, open-source HRTF datasets online (such as the MIT KEMAR or IRCAM Listen databases) packaged for FFmpeg.

You can use either: * A single multi-channel HRIR file where each channel represents the impulse response for a specific speaker position (FL, FR, FC, LFE, SL, SR, etc.). * Multiple individual mono HRIR files, one for each speaker position.

Step 2: The FFmpeg Command Syntax

Method A: Using a Single Multi-Channel HRIR File

If you have a single multi-channel WAV file containing your HRIR coefficients (e.g., hrir.wav with 6 channels for a 5.1 layout), use the following command:

ffmpeg -i input_51.mp4 -i hrir.wav -filter_complex "[0:a][1:a]headphone=map=FL|FR|FC|LFE|SL|SR[spatial]" -map 0:v -map "[spatial]" output.mp4

Method B: Using Separate Mono HRIR Files

If you have separate mono HRIR files for each speaker position, you must pass them as individual inputs to FFmpeg. Order them in the command and map them accordingly:

ffmpeg -i input_51.mp4 \
-i hrir_FL.wav -i hrir_FR.wav -i hrir_FC.wav -i hrir_LFE.wav -i hrir_SL.wav -i hrir_SR.wav \
-filter_complex "[0:a][1:a][2:a][3:a][4:a][5:a][6:a]headphone=map=FL|FR|FC|LFE|SL|SR[spatial]" \
-map 0:v -map "[spatial]" output.mp4

Parameter Breakdown

• [0:a]: Represents the multi-channel audio stream from your source video/audio file.
• [1:a] through [6:a]: Represent the HRIR inputs.
• headphone=: Activates the filter.
• map=: Defines which HRIR input corresponds to which virtual speaker. The positions are separated by pipe (|) characters. The positions must match the order of the HRIR inputs provided. Common position identifiers include:
  • FL: Front Left
  • FR: Front Right
  • FC: Front Center
  • LFE: Low Frequency Effects (Subwoofer)
  • SL: Surround Left
  • SR: Surround Right
  • BL: Back Left
  • BR: Back Right

Additional Settings

You can fine-tune the filter output using optional parameters inside the filter string:

• Gain: To avoid clipping or to boost quiet virtualizations, use the gain parameter (in dB).
• Type: Choose between time-domain (time) or frequency-domain (freq) processing. Frequency-domain is faster for long HRIR files.

Example with extra parameters:

ffmpeg -i input_51.mp4 -i hrir.wav -filter_complex "[0:a][1:a]headphone=map=FL|FR|FC|LFE|SL|SR:gain=3:type=freq[spatial]" -map 0:v -map "[spatial]" output.mp4