How to Create Binaural Audio with FFmpeg

This article explains how to use FFmpeg’s specialized audio spatialization filters to convert multi-channel audio into a 3D binaural stereo stream. You will learn the exact command-line syntax, how to map audio channels, and how to utilize Head-Related Impulse Response (HRIR) or SOFA files to achieve an immersive, spatialized headphone listening experience.

Understanding Binaural Filters in FFmpeg

While FFmpeg does not have a filter named literally binaural, it achieves binaural synthesis using two primary filters: headphone and sofalizer. These filters apply Head-Related Transfer Functions (HRTF) to multi-channel audio (like 5.1 or 7.1 surround sound), processing it so that a listener wearing standard stereo headphones perceives sound coming from specific directions in 3D space.

Method 1: Using the headphone Filter (HRIR WAV method)

The headphone filter is the most common way to create binaural audio in FFmpeg. It requires two inputs: your multi-channel audio track and a multi-channel HRIR (Head-Related Impulse Response) WAV file.

Step-by-Step Command

To convert a 5.1 surround sound file into a binaural stereo file, use the following command structure:

ffmpeg -i input_51.wav -i hrir.wav -filter_complex "[0:a][1:a]headphone=map=0|1|2|3|4|5[out]" -map "[out]" output_binaural.wav

Command Breakdown:

• -i input_51.wav: Your source audio (needs to be multi-channel, e.g., 6 channels for 5.1).
• -i hrir.wav: The HRIR coefficients file representing spatial positions.
• -filter_complex: Directs FFmpeg to process multiple inputs.
• [0:a][1:a]: Takes the audio from the first input (0) and the second input (1).
• headphone=map=0|1|2|3|4|5: The core filter. The map parameter assigns which channel of the HRIR file corresponds to which input channel. In this case, it maps Front Left, Front Right, Center, LFE, Back Left, and Back Right in order.
• [out]: The output label of the filter complex, mapped to the final output file.

Method 2: Using the sofalizer Filter (SOFA method)

If you have a .sofa (Spatially Oriented Format for Acoustics) file instead of an HRIR WAV file, you should use the sofalizer filter. This format contains highly accurate, standardized 3D audio measurements.

Step-by-Step Command

Run this command to apply a .sofa file directly to a multi-channel source:

ffmpeg -i input_51.wav -af "sofalizer=sofa=/path/to/spatial_profile.sofa" output_binaural.wav

Command Breakdown:

• -af: Applies a simple audio filter.
• sofalizer=sofa=/path/to/spatial_profile.sofa: Loads the specific SOFA profile to calculate the HRTF mathematical equations and apply them to the audio stream.

Advanced Customization (Optional)

You can customize the virtual speaker positions and gain inside the sofalizer filter:

ffmpeg -i input_51.wav -af "sofalizer=sofa=/path/to/spatial_profile.sofa:gain=3:rotation=90" output_binaural.wav

• gain: Adjusts the output volume in dB (useful if the binaural processing lowers the overall volume).
• rotation: Rotates the virtual listener’s head in degrees (0 to 360) to shift the soundstage perspective.