Create Multi-Language HLS Streams with FFmpeg

This article provides a step-by-step guide on how to use FFmpeg to create an HTTP Live Streaming (HLS) packager that supports multiple language audio tracks. You will learn how to take a video file and multiple separate audio files, encode them, and package them into a single HLS master playlist where viewers can select their preferred language during playback.

The FFmpeg Command

To create an HLS stream with separate, selectable audio tracks, you need to map your video and audio inputs correctly and use the -var_stream_map option in FFmpeg. This option groups the different audio streams and associates them with the video stream.

Here is the template command using a single video file and two separate audio files (English and Spanish):

ffmpeg -i video.mp4 -i audio_en.mp3 -i audio_es.mp3 \
-map 0:v:0 -map 1:a:0 -map 2:a:0 \
-c:v libx264 -c:a aac -b:a 128k \
-f hls \
-hls_time 10 \
-hls_playlist_type vod \
-master_pl_name master.m3u8 \
-var_stream_map "v:0,agroup:hd_audio a:0,agroup:hd_audio,language:eng,name:English,default:yes a:1,agroup:hd_audio,language:spa,name:Spanish" \
-hls_segment_filename "stream_%v/data%03d.ts" \
stream_%v.m3u8

Command Breakdown

• -i video.mp4 -i audio_en.mp3 -i audio_es.mp3: Imports the input video and the separate audio files.
• -map 0:v:0 -map 1:a:0 -map 2:a:0: Maps the first video stream from the first input (0), the first audio stream from the second input (1), and the first audio stream from the third input (2).
• -c:v libx264 -c:a aac: Encodes the video to H.264 and the audio tracks to AAC, which are standard codecs for HLS compatibility.
• -f hls: Specifies the output format as HLS.
• -hls_time 10: Splits the video and audio into 10-second segments.
• -master_pl_name master.m3u8: Tells FFmpeg to generate a master playlist file named master.m3u8 which binds the video and alternative audio tracks together.
• -var_stream_map: This is the key setting for multi-language HLS.
  • v:0,agroup:hd_audio: Associates the video stream (v:0) with an audio group named hd_audio.
  • a:0,agroup:hd_audio,language:eng,name:English,default:yes: Sets the first mapped audio stream as part of the hd_audio group, labels it as English, and sets it as the default track.
  • a:1,agroup:hd_audio,language:spa,name:Spanish: Sets the second mapped audio stream as part of the same hd_audio group and labels it as Spanish.
• -hls_segment_filename "stream_%v/data%03d.ts": Outputs the TS segments into separate directories (stream_0 for video, stream_1 for English audio, and stream_2 for Spanish audio) using the stream index placeholder (%v).
• stream_%v.m3u8: Generates individual variant playlist files for each stream.

Resulting Structure

After running the command, FFmpeg will generate the following directory structure:

• master.m3u8 (The main entry point for video players)
• stream_0.m3u8 (Video-only playlist)
• stream_1.m3u8 (English audio playlist)
• stream_2.m3u8 (Spanish audio playlist)
• stream_0/, stream_1/, and stream_2/ directories containing the .ts media segments.

When you load master.m3u8 into an HLS-compatible player (like Safari, VLC, or web players like Video.js and Hls.js), the player will display a menu allowing the user to switch seamlessly between English and Spanish audio while the video plays.