WebRTC Simulcast: Optimizing Multi-Party Video Routing

WebRTC Simulcast is a media negotiation technique that allows a single video client to transmit multiple resolutions and bitrates of the same video stream to a media server simultaneously. This article explores how WebRTC Simulcast works, its role in multi-party video conferencing, and how it enables Selective Forwarding Units (SFUs) to dynamically optimize video routing for participants with varying network conditions.

The Challenge of Multi-Party Video Calls

In a simple one-on-one WebRTC call, the two participants negotiate the best video resolution and bitrate based on their respective internet speeds and processing power. However, when multiple participants join a video conference, this peer-to-peer approach fails to scale.

If a central media server like a Selective Forwarding Unit (SFU) is used to route video, a new problem arises: different participants have different network conditions. Without Simulcast, the publisher must either send a single high-quality stream (which crashes the connection of participants with weak internet) or downgrade their video to the lowest common denominator (which ruins the experience for participants with fast internet).

What is WebRTC Simulcast?

WebRTC Simulcast solves this dilemma at the publisher level. Instead of sending a single video stream, the sender’s browser encodes and transmits multiple versions of the same video stream at the same time.

Typically, a publisher will send three distinct streams: * Low Quality: Low resolution (e.g., 180p) and low bitrate for participants with poor network connections or small display windows. * Medium Quality: Standard resolution (e.g., 360p or 480p) and moderate bitrate. * High Quality: High resolution (e.g., 720p or 1080p) and high bitrate for participants with excellent bandwidth and large active speaker windows.

How Simulcast Optimizes Multi-Party Video Routing

In a Simulcast setup, the Selective Forwarding Unit (SFU) acts as an intelligent router. It receives all three video qualities from the sender but does not decode or re-encode them, which keeps server CPU usage low. Instead, the SFU selectively routes the appropriate stream to each receiver based on two main factors:

1. Receiver Network Capacity

The SFU continuously monitors the packet loss and available bandwidth of every receiver. If a receiver’s network degrades, the SFU seamlessly switches their incoming stream from high quality to medium or low quality. When the network recovers, the SFU switches them back to the high-quality stream.

2. Application Layout and UI Needs

In large meetings, not everyone needs to be viewed in high definition. If a participant is minimized in a grid or thumbnail view, the SFU will route the low-quality stream to save download bandwidth. If a participant becomes the active speaker and their video is maximized, the SFU instantly upgrades their stream to high quality.

Benefits of WebRTC Simulcast

Implementing WebRTC Simulcast offers several key advantages for multi-party video applications: