How the JavaScript MediaRecorder API Encodes Media

The MediaRecorder API provides a standardized interface for capturing real-time media streams—such as microphone input, webcam feeds, or screen shares—and encoding them directly into playable audio or video files in the browser. This article explains the underlying mechanism of the API, detailing how it ingests a MediaStream, processes raw data through browser-level codecs, collects encoded binary segments, and outputs a cohesive media file formatted for playback or download.

1. Ingesting the MediaStream

The recording pipeline begins with a MediaStream instance. This stream is typically acquired from device hardware via navigator.mediaDevices.getUserMedia(), screen capture via getDisplayMedia(), or generated dynamically from an HTML5 <canvas> or Web Audio context.

A MediaStream consists of one or more MediaStreamTrack objects representing individual audio and video channels. The MediaRecorder receives this stream as its primary input upon instantiation:

const stream = await navigator.mediaDevices.getUserMedia({ video: true, audio: true });
const recorder = new MediaRecorder(stream, { mimeType: 'video/webm; codecs=vp9,opus' });

2. Codec Selection and Configuration

Before encoding starts, the browser determines the target container format and compression codecs. Developers can specify these parameters using the mimeType option. To ensure cross-browser compatibility, codec availability can be verified beforehand using MediaRecorder.isTypeSupported():

const isSupported = MediaRecorder.isTypeSupported('video/mp4; codecs=avc1,mp4a');

Commonly supported codecs include VP8, VP9, and AV1 for video, along with Opus for audio (typically packaged within a WebM container), as well as H.264/AVC and AAC (within an MP4 container).

3. Real-Time Encoding and Chunk Generation

When recorder.start() is invoked, the browser routes raw frame data and audio samples from the MediaStreamTrack inputs to its internal hardware- or software-accelerated encoders.

  1. Frame & Sample Compression: Video frames and audio PCM buffers are compressed into keyframes, delta frames, and encoded audio packets according to the configured codec.
  2. Container Multiplexing (Muxing): The encoder multiplexes the compressed audio and video tracks into a container format (e.g., WebM or MP4), embedding initialization headers, metadata, and timestamps.
  3. Timeslice Buffering: If a timeslice argument (in milliseconds) is passed to recorder.start(timeslice), the recorder emits data periodically. Otherwise, data is held in internal memory until recorder.stop() or recorder.requestData() is called.

4. Collecting Binary Data via Events

As the encoded data becomes available, the MediaRecorder triggers the dataavailable event. The event payload (event.data) contains a Blob containing the latest slice of binary data:

const recordedChunks = [];

recorder.ondataavailable = (event) => {
  if (event.data && event.data.size > 0) {
    recordedChunks.push(event.data);
  }
};

5. Assembling the Final Playable File

Once recording concludes, the stop event fires. The collected binary chunks are merged into a single consolidated Blob with the matching MIME type. This resulting file is fully muxed and immediately playable by standard media players:

recorder.onstop = () => {
  const completeBlob = new Blob(recordedChunks, { type: 'video/webm' });
  const videoURL = URL.createObjectURL(completeBlob);
  
  // Assign to a video element or create a download link
  const videoElement = document.querySelector('video');
  videoElement.src = videoURL;
};

// Stop recording
recorder.stop();

By leveraging browser-native codecs and handling stream synchronization internally, the MediaRecorder API delivers an efficient, low-overhead pipeline for producing standard media files directly on the client side.