WebRTC Insertable Streams for End-to-End Encryption

This article explains how the WebRTC Insertable Streams API (also known as WebRTC Encoded Transform) enables true end-to-end encryption (E2EE) in JavaScript applications. It breaks down the limitation of standard WebRTC encryption in multi-party conferences, describes how insertable streams intercept encoded media frames before transmission, and demonstrates how custom cryptographic layers can be implemented in a browser environment to secure media from intermediate routing servers.

The WebRTC Encryption Challenge in Multi-Party Calls

Standard WebRTC natively uses DTLS (Datagram Transport Layer Security) and SRTP (Secure Real-time Transport Protocol) to encrypt voice and video traffic. While this provides secure peer-to-peer communication, multi-party video conferencing typically relies on a central Selective Forwarding Unit (SFU) to scale efficiently.

In a traditional SFU architecture, DTLS-SRTP terminates at the media server. The SFU decrypts incoming media packets to inspect metadata (like spatial or temporal layers) and re-encrypts them before forwarding to other participants. As a result, the server operator has access to the unencrypted audio and video, preventing true end-to-end encryption.

How Insertable Streams Solve the Problem

The Insertable Streams specification introduces a mechanism to intercept raw, encoded media frames within the browser pipeline. Instead of allowing the browser to immediately packetize and transmit the output of a video or audio codec, the API exposes the encoded frames as a ReadableStream and WritableStream via a TransformStream.

This architecture places a processing hook directly between: 1. The Encoder and Packetizer on the sender’s side. 2. The Depacketizer and Decoder on the receiver’s side.

By exposing these streams in JavaScript or Web Workers, developers can insert custom cryptographic operations directly on the encoded frame data before the browser applies standard transport-layer encryption.

The End-to-End Encryption Workflow

Implementing E2EE with Insertable Streams follows a four-step pipeline:

Sender:    [Media Stream] -> [Encoder] -> [Custom Encryption] -> [SRTP/DTLS] -> (Network)
                                                                                  |
Receiver:  [Media Stream] <- [Decoder] <- [Custom Decryption] <- [SRTP/DTLS] <-- (Network)

1. Frame Interception (Sender)

When a sender captures audio or video, the browser encodes the raw media into encoded frames (such as VP8, VP9, AV1, or Opus). Using the RTCRtpSender.createEncodedStreams() method (or RTCRtpScriptTransform), the application intercepts the stream of encoded chunks (RTCEncodedAudioFrame or RTCEncodedVideoFrame).

2. Custom Payload Encryption

Inside a dedicated TransformStream (typically running inside a Web Worker to avoid blocking the main UI thread), the application encrypts the frame’s payload using standard symmetric ciphers, such as AES-GCM or AES-CTR:

• Payload Encryption: The actual compressed audio/video bytes are encrypted using a shared conference key or double-ratchet session keys.
• Metadata Preservation: Certain metadata fields (such as frame type, keyframe indicators, and temporal layer indices) are left unencrypted or authenticated via Additional Authenticated Data (AAD). This allows the SFU to route packets correctly without knowing the contents of the media.

3. Transport and Routing

The custom-encrypted frame is passed back to the WebRTC pipeline, which applies standard DTLS-SRTP encryption before transmission over the network. When the SFU receives the stream, it terminates the DTLS-SRTP layer, inspects the unencrypted routing metadata, and forwards the packet. Because the underlying payload remains protected by the application-layer encryption, the SFU cannot access the raw media.

4. Frame Interception and Decryption (Receiver)

On the receiving client, the incoming packet passes through the standard DTLS-SRTP decryption layer. Before the frame reaches the browser’s hardware or software decoder:

• The receiver intercepts the stream using RTCRtpReceiver.createEncodedStreams() or RTCRtpScriptTransform.
• The custom decryption transform uses the matching key to decrypt the payload.
• The decrypted RTCEncodedFrame is forwarded to the native decoder, which renders the media onto a <video> or <audio> element.

Key Advantages of Insertable Streams

• Zero Infrastructure Changes: SFUs do not need to support custom encryption protocols; they continue routing packets based on standard RTP headers and exposed frame metadata.
• Worker-Based Performance: Using WebRTC Encoded Transform with Web Workers ensures cryptographic operations run off the main thread, maintaining low latency and high frame rates.
• Flexible Key Management: Applications can implement custom key exchange protocols, such as MLS (Messaging Layer Security) or Signal’s Double Ratchet, independent of WebRTC transport protocols.