Switch Video Streams with FFmpeg Streamselect

This guide explains how to dynamically switch between multiple input video streams in FFmpeg using the streamselect filter. You will learn the core parameters of streamselect, how to combine it with the sendcmd filter to schedule stream transitions at specific timestamps, and how to control stream switching in real time using ZeroMQ.

Understanding the streamselect Filter

The streamselect filter takes \(N\) input streams and maps one or more of them to the output. By default, it operates statically, but it supports runtime commands to change the active input index without interrupting the output stream.

Basic parameters include:

  • inputs: The number of input streams (default is 2).
  • map: The 0-based index of the input stream(s) to route to the output.

Because streamselect does not evaluate dynamic expressions on its own, dynamic switching requires sending commands to it during processing. This is typically achieved using either the sendcmd filter (for pre-planned, time-based switching) or the zmq filter (for interactive, external control).

Scheduled Switching with sendcmd

The sendcmd filter can dispatch commands to streamselect at predefined timestamps.

Command Syntax

To switch streams, format the command string as: [timestamp] streamselect map [input_index]

Example: Timed Switching

The following command takes two inputs (cam1.mp4 and cam2.mp4). It starts on cam1.mp4 (index 0), switches to cam2.mp4 (index 1) at 5 seconds, and switches back to cam1.mp4 at 10 seconds.

ffmpeg -i cam1.mp4 -i cam2.mp4 -filter_complex \
"[0:v]sendcmd=c='5.0 streamselect map 1; 10.0 streamselect map 0'[cmd0]; \
 [cmd0][1:v]streamselect=inputs=2:map=0[outv]" \
-map "[outv]" -c:v libx264 output.mp4

In this command:

  1. sendcmd intercepts stream 0 and injects switching instructions at seconds 5.0 and 10.0.
  2. Both the processed stream [cmd0] and the second input [1:v] enter streamselect.
  3. streamselect=inputs=2:map=0 defaults to the first input until the command updates the map value.

Using an External Command File

For complex switching schedules, store the instructions in a separate text file (e.g., commands.txt):

5.0 streamselect map 1;
10.0 streamselect map 0;
15.0 streamselect map 1;

Load the file using the f parameter:

ffmpeg -i cam1.mp4 -i cam2.mp4 -filter_complex \
"[0:v]sendcmd=f=commands.txt[cmd0]; \
 [cmd0][1:v]streamselect=inputs=2:map=0[outv]" \
-map "[outv]" -c:v libx264 output.mp4

Real-Time Switching with ZeroMQ (zmq)

If you are streaming live and need to switch inputs on demand, combine streamselect with the zmq filter to receive commands over a network socket.

ffmpeg -i live1.flv -i live2.flv -filter_complex \
"[0:v]zmq[cmd0]; \
 [cmd0][1:v]streamselect=inputs=2:map=0[outv]" \
-map "[outv]" -c:v libx264 -f flv rtmp://destination/live

To trigger a switch while FFmpeg is running, send a message to the listening socket using the zmqsend utility or a custom script:

echo "streamselect map 1" | zmqsend

Key Considerations

  • Stream Parameters: All input streams should share the same resolution, pixel format, and frame rate. If they differ, normalize them beforehand using scale, fps, and format filters.
  • Audio Synchronization: The streamselect filter also supports audio streams (astreamselect). If you are switching video and audio together, route both through their respective streamselect and astreamselect filters synchronized with matching timestamps.