WebRTC Insertable Streams API for E2EE in SFUs

This article explains the WebRTC Insertable Streams API, a web standard that allows developers to access and manipulate raw media data in real-time communication pipelines. You will learn how this API operates, the security limitations of traditional Selective Forwarding Units (SFUs), and the exact mechanism by which Insertable Streams enable true end-to-end encryption (E2EE) while preserving the efficiency of modern video conferencing architectures.

What is the WebRTC Insertable Streams API?

The WebRTC Insertable Streams API (sometimes referred to as SFrame transform or WebRTC encoded transforms) is a browser API that exposes the media pipeline of a WebRTC connection.

In a standard WebRTC connection, the browser automatically handles the acquisition of media (camera/microphone), encoding, packetization, encryption (via SRTP/DTLS), and transmission. Developers historically had no access to the encoded media frames before they were sent over the network.

Insertable Streams break this closed pipeline by introducing Transform Streams (using the Streams API standard) between the encoder and the packetizer on the sender side, and between the depacketizer and the decoder on the receiver side. This allows developers to “insert” custom JavaScript or WebAssembly code to inspect or modify encoded video frames and audio chunks in transit.

The SFU Security Dilemma

To understand why Insertable Streams are revolutionary, it is necessary to look at how modern multi-party video conferencing works.

Most modern platforms use a Selective Forwarding Unit (SFU). Instead of every participant sending their video to everyone else (mesh network) or a central server decoding and mixing all videos into one stream (MCU), an SFU acts as a smart router. It receives media streams from each participant and forwards them to the other participants without decoding the video.

However, traditional WebRTC encryption (DTLS-SRTP) is hop-by-hop: 1. The sender encrypts the media for the SFU. 2. The SFU decrypts the media to read the packet headers and route the data. 3. The SFU re-encrypts the media for each receiver.

Because the SFU decrypts the media, the server hosting the SFU has access to the raw video and audio. If the SFU server is compromised, malicious actors can spy on the call. True End-to-End Encryption (E2EE) was previously impossible with SFUs because the server required access to the transport layer.

How Insertable Streams Enable True E2EE

The WebRTC Insertable Streams API uniquely solves this problem by allowing application-level encryption on top of the standard transport-level encryption.

Here is the step-by-step process of how E2EE is achieved using Insertable Streams in an SFU architecture:

1. Sender-Side Encryption (The Transform)

Before the encoded video frame or audio chunk is sent to the WebRTC transport layer, the Insertable Streams API intercepts it. The developer’s code encrypts the raw media payload using a custom key shared only among the participants (using algorithms like AES-GCM). Crucially, the metadata required for network routing (such as RTP headers) is left unencrypted.

The double-encrypted packet is sent to the SFU. The SFU decrypts the outer transport-level encryption (DTLS-SRTP) to read the routing metadata. However, the actual media payload remains encrypted with the secret application-level key. Because the SFU does not have this key, it cannot decrypt or view the media. It simply forwards the encrypted payload to the receivers.

3. Receiver-Side Decryption

When the packet arrives at the receiver’s browser, the standard WebRTC transport decryption is stripped off. Before the packet is sent to the browser’s video/audio decoder, the Insertable Streams API intercepts the encoded frame. The receiver’s application code uses the shared key to decrypt the media payload. The decrypted, standard encoded frame is then passed to the decoder and rendered on the screen.

Key Benefits of this Approach

Zero SFU Overhead: Because the SFU does not need to decrypt or re-encrypt the media payload, it remains highly efficient, handling routing tasks with minimal CPU usage.
Trustless Infrastructure: Service providers can host SFUs on public clouds or third-party servers without compromising user privacy, as the server never possesses the keys to decrypt the media.
Flexibility: Developers can implement their own key exchange mechanisms (such as MLS or Signal protocol) and encryption algorithms without waiting for browser vendors to native-standardize them.