Stream Multiple Audio Tracks Over RTMP with FFmpeg

This guide explains how to stream multiple audio tracks over RTMP using FFmpeg. You will learn the exact FFmpeg commands needed to map multiple audio inputs, configure the output stream, and handle the inherent limitations of the RTMP protocol when dealing with multiple audio sources.

The Challenge with RTMP and Multiple Audio Tracks

The Real-Time Messaging Protocol (RTMP) traditionally relies on the FLV (Flash Video) container format. Standard FLV officially supports only a single video track and a single audio track.

If you attempt to stream multiple distinct audio tracks (such as different languages) over standard RTMP, most ingestion servers and video players will only play the first audio track and discard the rest.

To overcome this, you have two primary methods: mapping multiple tracks directly (for modern servers that support it) or merging multiple audio sources into a single multi-channel track.

Method 1: Mapping Multiple Audio Streams (Standard Mapping)

If your RTMP destination or ingestion server (such as certain private media servers or proprietary players) supports multi-track RTMP, you can use FFmpeg’s -map option to send multiple audio streams.

Run the following command to map a video source and two separate audio sources:

ffmpeg -re -i video.mp4 -i audio_en.wav -i audio_es.wav \
-map 0:v:0 -map 1:a:0 -map 2:a:0 \
-c:v libx264 -c:a aac -b:a 128k \
-f flv rtmp://your-rtmp-server-url/live/stream_key

Command Breakdown:

• -re: Tells FFmpeg to read the input in real-time (essential for live streaming).
• -i: Specifies the input files (video, English audio, Spanish audio).
• -map 0:v:0: Maps the first video stream from the first input.
• -map 1:a:0: Maps the first audio stream from the second input (English).
• -map 2:a:0: Maps the first audio stream from the third input (Spanish).
• -c:v libx264 -c:a aac: Encodes the video to H.264 and both audio streams to AAC.
• -f flv: Forces the output format to FLV for RTMP compatibility.

Method 2: Merging Audio Tracks into a Single Multi-Channel Stream

Because many RTMP players only recognize a single audio track, the most reliable workaround is to merge two mono audio tracks into a single stereo stream (Left and Right channels) or combine them into a 5.1 surround sound stream.

This is highly effective for broadcasting dual-language commentary (e.g., English on the Left, Spanish on the Right).

Run this command to merge two separate audio inputs into a single stereo track:

ffmpeg -re -i video.mp4 -i audio_left.wav -i audio_right.wav \
-filter_complex "[1:a][2:a]join=inputs=2:channel_layout=stereo[aout]" \
-map 0:v:0 -map "[aout]" \
-c:v libx264 -c:a aac \
-f flv rtmp://your-rtmp-server-url/live/stream_key

Command Breakdown:

• -filter_complex: Initiates a complex filtergraph.
• [1:a][2:a]join=inputs=2:channel_layout=stereo[aout]: Takes the audio from the second and third inputs, joins them together into a single stereo output, and labels it [aout].
• -map "[aout]": Maps the newly joined stereo audio track to the RTMP output.

Recommended Alternatives for True Multi-Track Streaming

If your target audience needs to dynamically switch between audio tracks (such as selecting different languages in a player menu), RTMP is not the ideal protocol. Instead, consider using:

1. SRT (Secure Reliable Transport): Supports native multi-track MPEG-TS containers.
2. HLS (HTTP Live Streaming): Allows you to deliver multiple audio playlist files (.m3u8) grouped under a master playlist, enabling native language selection in modern players.