How to Group Video and Audio in FFmpeg DASH

When streaming using the Dynamic Adaptive Streaming over HTTP (DASH) protocol, properly grouping your video and audio streams into adaptation sets is crucial for client players to choose the right quality and language. This article provides a straightforward guide on how to configure adaptation sets in FFmpeg using the -adaptation_sets option, ensuring your video and audio tracks are correctly mapped and organized for seamless adaptive playback.

Understanding DASH Adaptation Sets in FFmpeg

In DASH, an Adaptation Set contains a set of interchangeable streams (representations) that the player can switch between based on network conditions (e.g., different video resolutions) or user preferences (e.g., different audio languages).

To define these sets in FFmpeg, you use the -adaptation_sets option within the dash muxer. The basic syntax is:

-adaptation_sets "id=SET_ID,streams=STREAM_SELECTORS"

id: A unique integer identifying the adaptation set.
streams: A space-separated list of output stream indexes or stream type specifiers (like v for video and a for audio) assigned to this set.

Scenario 1: Grouping All Videos and All Audios (Basic)

If you have multiple video qualities (e.g., 1080p, 720p, 480p) and one audio stream, you want all video streams in one adaptation set and the audio stream in another.

Use the following command helper:

ffmpeg -i input.mp4 \
  -map 0:v -b:v:0 5000k -s:v:0 1920x1080 \
  -map 0:v -b:v:1 3000k -s:v:1 1280x720 \
  -map 0:a -c:a aac \
  -f dash \
  -adaptation_sets "id=0,streams=v id=1,streams=a" \
  manifest.mpd

id=0,streams=v: Groups all mapped video streams (v) into Adaptation Set 0.
id=1,streams=a: Groups all mapped audio streams (a) into Adaptation Set 1.

Scenario 2: Grouping by Specific Stream Indexes (Advanced)

When dealing with multiple audio languages or specific stream assignments, grouping by stream type (v or a) is not enough. You must reference the output streams by their zero-based index in the order they are mapped.

Consider this stream mapping order: * Output Stream 0: Video (1080p) * Output Stream 1: Video (720p) * Output Stream 2: Audio (English) * Output Stream 3: Audio (Spanish)

To group the two videos together, and keep the English and Spanish audio tracks in separate, selectable adaptation sets, configure the command like this:

ffmpeg -i input.mkv \
  -map 0:v:0 -s:v:0 1920x1080 \
  -map 0:v:0 -s:v:1 1280x720 \
  -map 0:a:0 -c:a:0 aac \
  -map 0:a:1 -c:a:1 aac \
  -f dash \
  -adaptation_sets "id=0,streams=0,1 id=1,streams=2 id=2,streams=3" \
  manifest.mpd

id=0,streams=0,1: Places the 1080p and 720p video streams into the first set. The player will dynamically switch between these based on bandwidth.
id=1,streams=2: Places the English audio stream into its own set.
id=2,streams=3: Places the Spanish audio stream into its own set, allowing the user to toggle languages.

Best Practices for Adaptation Sets

Always Match Output Indexes: The numbers in streams=0,1 refer to the order of -map arguments in your FFmpeg command, not the track IDs of the input file.
Double Quotes: Always wrap the -adaptation_sets argument in double quotes to prevent command-line parsing errors, especially when using spaces to separate multiple sets.
Set Descriptor Attributes: If you need to specify languages or roles, you can append them inside the adaptation set string (e.g., id=1,lang=eng,streams=2).