FFmpeg DASH Multiple Audio Track Adaptation Sets

This article provides a practical guide on configuring multiple audio tracks into distinct adaptation sets within a Dynamic Adaptive Streaming over HTTP (DASH) manifest using FFmpeg. It covers the core -adaptation_sets option, stream mapping techniques, language metadata assignment, and command-line examples necessary to produce compliant multi-track audio manifests.

The -adaptation_sets Option

In FFmpeg's DASH muxer (-f dash), the primary option used to define manifest structure is -adaptation_sets. By default, FFmpeg groups all video streams into one Adaptation Set and all audio streams into another. For multi-language or multi-track audio, separate Adaptation Sets are required so DASH clients can switch between languages independently.

The syntax for the option is:

-adaptation_sets "id=SET_ID,streams=STREAM_SPECIFIER id=SET_ID,streams=STREAM_SPECIFIER ..."
  • id: A unique numerical identifier for the Adaptation Set within the MPD (Media Presentation Description).
  • streams: A comma-separated list of output stream indices or stream specifiers assigned to that Adaptation Set.

Mapping Multiple Audio Tracks

Before grouping streams into adaptation sets, multiple audio streams must be explicitly mapped and encoded using the -map option. Each stream can be assigned language metadata using -metadata:s:a:X language=CODE, which DASH clients use to present language options in the player UI.

For example, to map one video track and two stereo audio tracks (English and Spanish):

-map 0:v:0 -map 0:a:0 -map 0:a:1 \
-c:v libx264 -c:a aac \
-metadata:s:a:0 language=eng \
-metadata:s:a:1 language=spa

Assigning Audio Tracks to Separate Adaptation Sets

To create separate Adaptation Sets for each audio language, declare a unique id and specify the stream index for each audio track in the -adaptation_sets string.

Example Configuration

ffmpeg -i input.mkv \
  -map 0:v:0 -map 0:a:0 -map 0:a:1 \
  -c:v libx264 -b:v 3000k \
  -c:a:0 aac -b:a:0 128k -metadata:s:a:0 language=eng \
  -c:a:1 aac -b:a:1 128k -metadata:s:a:1 language=spa \
  -f dash \
  -adaptation_sets "id=0,streams=v id=1,streams=1 id=2,streams=2" \
  manifest.mpd

In this command:

  • id=0,streams=v assigns all video streams (stream 0) to Adaptation Set 0.
  • id=1,streams=1 assigns the English audio stream (stream 1) to Adaptation Set 1.
  • id=2,streams=2 assigns the Spanish audio stream (stream 2) to Adaptation Set 2.

Grouping Audio Representations of the Same Language

If providing multiple bitrates or codecs for the same language (such as high-bitrate and low-bitrate English), group those specific streams under the same Adaptation Set ID:

-map 0:v:0 -map 0:a:0 -map 0:a:0 -map 0:a:1 \
-c:v libx264 \
-c:a:0 aac -b:a:0 192k -metadata:s:a:0 language=eng \
-c:a:1 aac -b:a:1 96k  -metadata:s:a:1 language=eng \
-c:a:2 aac -b:a:2 128k -metadata:s:a:2 language=spa \
-f dash \
-adaptation_sets "id=0,streams=0 id=1,streams=1,2 id=2,streams=3" \
manifest.mpd

Here, stream 1 and stream 2 are grouped into Adaptation Set 1, allowing the client player to adaptively switch audio quality for English while keeping Spanish isolated in Adaptation Set 2.

Defining Stream Roles and Descriptors

The -adaptation_sets parameter also accepts descriptor attributes such as role to specify track types (e.g., main, alternate, commentary, description):

-adaptation_sets "id=0,streams=v id=1,streams=1,role=main id=2,streams=2,role=description"

This adds a <Role schemeIdUri="urn:mpeg:dash:role:2011" value="..."/> element inside the resulting <AdaptationSet> in the manifest, alerting the player of accessibility or commentary tracks.