Stream Multiple Audio Tracks Over RTMP with FFmpeg

Streaming multiple audio tracks over RTMP using FFmpeg requires either packing multiple audio streams into an enhanced RTMP connection or merging separate audio feeds into distinct channels of a single multichannel track. Because legacy RTMP and standard FLV containers were originally designed to hold only a single audio stream, standard implementations reject secondary audio tracks. This guide details how to configure FFmpeg to send multiple audio tracks using both standard multi-channel multiplexing and modern multi-track stream mapping.

Understanding the RTMP Limitation

The original Adobe RTMP specification uses the FLV container format, which restricts the media stream to one video track and one audio track. To bypass this restriction, you must use one of two methods:

  1. Multi-Track Mapping (Enhanced RTMP): Sending multiple distinct audio streams (0:a:0, 0:a:1, etc.). This requires an ingest server that supports Enhanced RTMP or multi-track FLV extensions.
  2. Channel Merging (Channel Packing): Combining separate audio sources into a single multi-channel track (e.g., 4-channel quadraphonic or 5.1 surround sound). This works on all standard RTMP servers, allowing the receiving end to demux or split the channels.

Method 1: Mapping Multiple Discrete Audio Streams

If your RTMP ingest server supports multi-track audio (such as Enhanced RTMP v2 or custom media servers like SRS, Red5, or NGINX-RTMP configured for multi-track payloads), you can use FFmpeg's -map option to assign separate audio inputs.

Command Example:

ffmpeg -re -i video_source.mp4 -i english.wav -i spanish.wav \
  -map 0:v:0 \
  -map 1:a:0 \
  -map 2:a:0 \
  -c:v libx264 -preset veryfast -b:v 4500k \
  -c:a:0 aac -b:a:0 160k \
  -c:a:1 aac -b:a:1 160k \
  -f flv rtmp://your-server.com/live/streamkey

Command Breakdown:

  • -re: Reads the input at the native frame rate (essential for live streaming).
  • -map 0:v:0: Maps the video stream from the first input.
  • -map 1:a:0: Maps the audio stream from the second input as the primary audio track.
  • -map 2:a:0: Maps the audio stream from the third input as the secondary audio track.
  • -c:a:0 and -c:a:1: Explicitly defines the codec and bitrate for each individual audio stream.
  • -f flv: Encapsulates the output in the FLV container required by RTMP.

Method 2: Merging Tracks into a Single Multichannel Stream

If your destination server does not support Enhanced RTMP and drops secondary audio streams, merge the separate stereo or mono tracks into one multi-channel track. For example, you can merge two stereo inputs into a single 4-channel audio stream.

Command Example:

ffmpeg -re -i video_source.mp4 -i track1.wav -i track2.wav \
  -filter_complex "[1:a][2:a]amerge=inputs=2[aout]" \
  -map 0:v:0 \
  -map "[aout]" \
  -c:v libx264 -preset veryfast -b:v 4500k \
  -c:a aac -ac 4 -b:a 320k \
  -f flv rtmp://your-server.com/live/streamkey

Command Breakdown:

  • -filter_complex "[1:a][2:a]amerge=inputs=2[aout]": Combines the audio streams from input 1 and input 2 into a single multi-channel output stream named [aout].
  • -map "[aout]": Selects the merged audio stream for the final output.
  • -ac 4: Sets the audio channel count to 4 (Channels 1 and 2 represent the first input, while Channels 3 and 4 represent the second).

Method 3: Streaming Multi-Track from a Single Multi-Audio File

If your source file already contains multiple audio tracks, inspect the stream order first using ffprobe:

ffprobe -i input.mp4

Identify the stream indexes (e.g., Stream #0:0 is video, Stream #0:1 is Track 1, Stream #0:2 is Track 2). Stream both tracks over RTMP using:

ffmpeg -re -i input.mp4 \
  -map 0:v:0 \
  -map 0:a:0 \
  -map 0:a:1 \
  -c:v copy \
  -c:a aac -b:a 160k \
  -f flv rtmp://your-server.com/live/streamkey

Best Practices

  • Codec Support: AAC is the universally accepted standard for audio over RTMP. Avoid using MP3 or Opus unless your target server explicitly supports non-standard FLV codecs.
  • Synchronization: Ensure all audio inputs have identical sample rates (such as -ar 48000) to prevent audio drift during the live stream.
  • Server Verification: If the receiving platform only plays one track, verify server-side logs to confirm whether secondary streams are discarded upon ingestion.