Convert 360 Ambisonic Audio to Binaural in FFmpeg

This guide explains how to extract 4-channel first-order Ambisonics (AmbiX) spatial audio from a 360-degree video file and decode it into a standard binaural stereo track using FFmpeg. By decoding the spherical soundfield into virtual speaker positions and processing them through a Head-Related Transfer Function (HRTF) using FFmpeg’s sofalizer filter, you can produce a realistic 3D headphone listening experience from any Ambisonic source.

Step 1: Identify the Spatial Audio Stream

Most 360-degree videos (such as YouTube 360 or spatial video from omnidirectional cameras) store audio in first-order Ambisonics (FOA) using the AmbiX format (ACN channel ordering, SN3D normalization) across four channels:

  • Channel 0 (W): Omni-directional pressure
  • Channel 1 (Y): Left / Right dipole
  • Channel 2 (Z): Up / Down dipole
  • Channel 3 (X): Front / Back dipole

Check the audio stream details of your file using ffprobe:

ffprobe -i input_360.mp4

Look for an audio stream showing 4 channels or quad layout.

Step 2: Obtain an HRTF Profile (.sofa)

FFmpeg uses the sofalizer audio filter to create binaural audio. This filter requires a SOFA (Spatially Oriented Format for Acoustics) file containing HRTF measurements.

You can download standard, free SOFA measurement files (such as ClubFritz12.sofa or public IRCAM datasets) directly from the official SOFA Conventions repository or open acoustic research archives. Place the .sofa file in your working directory.

Step 3: Decode Ambisonics to Virtual Speakers and Apply Binauralization

Because Ambisonic channels represent directional coefficients rather than discrete speaker feeds, they must first be decoded into a virtual loudspeaker array (such as an 8-point virtual cube around the listener) before passing through the HRTF filter.

Run the following command to decode 4-channel AmbiX to an 8-speaker virtual array via matrix math, then render it to binaural stereo:

ffmpeg -i input_360.mp4 -vn -filter_complex \
"[0:a]pan=8c|\
c0=0.3535*c0 + 0.2041*c1 + 0.2041*c2 + 0.2041*c3|\
c1=0.3535*c0 - 0.2041*c1 + 0.2041*c2 + 0.2041*c3|\
c2=0.3535*c0 + 0.2041*c1 - 0.2041*c2 + 0.2041*c3|\
c3=0.3535*c0 - 0.2041*c1 - 0.2041*c2 + 0.2041*c3|\
c4=0.3535*c0 + 0.2041*c1 + 0.2041*c2 - 0.2041*c3|\
c5=0.3535*c0 - 0.2041*c1 + 0.2041*c2 - 0.2041*c3|\
c6=0.3535*c0 + 0.2041*c1 - 0.2041*c2 - 0.2041*c3|\
c7=0.3535*c0 - 0.2041*c1 - 0.2041*c2 - 0.2041*c3,\
sofalizer=sofa=ClubFritz12.sofa:type=freq:speakers=45 35|315 35|135 35|225 35|45 -35|315 -35|135 -35|225 -35[out]" \
-map "[out]" -c:a pcm_s16le output_binaural.wav

How the Command Works:

  • -vn: Disables video processing to speed up extraction.
  • pan=8c|...: Projects the 4-channel B-format signal into an 8-point cube representing virtual speakers at azimuths \(45^\circ\), \(135^\circ\), \(225^\circ\), and \(315^\circ\), each elevated at \(+35^\circ\) and \(-35^\circ\).
  • sofalizer=sofa=...: Loads the HRTF dataset.
  • speakers=...: Defines the exact coordinates (azimuth and elevation) for each of the 8 virtual speakers to match the decoded signals.
  • type=freq: Uses frequency-domain processing for faster performance.
  • -c:a pcm_s16le: Outputs an uncompressed 16-bit stereo WAV file.

Step 4: (Optional) Mux the Binaural Track Back to Video

If you want to create a standard 2D flat video containing the binaural audio:

ffmpeg -i input_360.mp4 -i output_binaural.wav -map 0:v -map 1:a -c:v copy -c:a aac -b:a 320k output_video_binaural.mp4

This retains the original video stream without re-encoding while replacing the 4-channel spatial track with a headphone-ready AAC stereo track.