Firefox Integration with Windows DirectComposition APIs

This article explores how the Mozilla Firefox web browser integrates with the DirectComposition API in Windows 10 and Windows 11 to utilize hardware overlays. It covers the architectural role of Firefox's WebRender engine, the mechanics of delegating video composition to the Windows Desktop Window Manager, and how this integration optimizes rendering performance and power efficiency.

The Role of WebRender

Firefox uses a Rust-based rendering engine called WebRender to handle page rendering. On Windows 10 and Windows 11, WebRender interfaces directly with DirectComposition, a native Windows API designed for high-performance bitmap composition.

Instead of drawing the entire browser window—including UI, text, and video—into a single monolithic buffer, WebRender uses DirectComposition to break the webpage into distinct visual layers. This separation is the foundational requirement for enabling hardware overlays.

What are Hardware Overlays?

Hardware overlays, implemented in modern Windows as Multiplane Overlays (MPO), allow the GPU's display controller to combine multiple image buffers directly as the image is sent to the monitor (scan-out).

Without hardware overlays, the GPU must use its 3D rendering engine to copy, scale, and convert the color space of a video frame, merge it with the browser window, and write it to a final backbuffer. With hardware overlays, the display controller reads the raw video buffer and the browser UI buffer separately, merging them on the fly. This bypasses the 3D engine entirely, drastically reducing GPU power consumption and memory bandwidth.

The DirectComposition Pipeline in Firefox

When a user plays a video in Firefox on Windows 10 or 11, the browser executes a specific pipeline to promote that video to a hardware overlay:

  1. Hardware Decoding: Firefox's media pipeline decodes the video stream using DirectX Video Acceleration (DXVA). The output is stored in a Direct3D texture, typically in a YUV color format like NV12 (for 8-bit video) or P010 (for 10-bit HDR video).
  2. SwapChain Creation: Instead of converting this YUV texture to RGB and drawing it into the main browser window, Firefox creates a dedicated DXGI SwapChain specifically for the video element.
  3. Visual Tree Construction: Firefox uses the DirectComposition API to build a "visual tree." It creates a DirectComposition visual surface for the main browser content and a separate visual surface bound to the video's DXGI SwapChain.
  4. Layer Positioning: If the video is underneath HTML elements (like custom play buttons or subtitles), Firefox places the video visual behind the UI visual using DirectComposition's Z-ordering capabilities.
  5. DWM Handoff: Firefox commits the DirectComposition visual tree to the Windows Desktop Window Manager (DWM), handing over control of the final composition to the operating system.

Execution by the Desktop Window Manager

Once the DWM receives the visual tree and the separated SwapChains from Firefox, it determines if the video can be promoted to a hardware overlay. The DWM queries the graphics driver for Multiplane Overlay support.

If the GPU and connected display support MPO, the DWM assigns the Firefox video SwapChain to a dedicated hardware overlay plane. During the monitor's refresh cycle, the GPU's display controller directly fetches the NV12 or P010 video frame. It performs the necessary YUV-to-RGB color space conversion and scaling in dedicated fixed-function hardware, blending it with the Firefox UI layer at the exact coordinates specified by the DirectComposition visual tree.

Handling Fallbacks and Limitations

Hardware overlays have strict hardware limitations. A GPU only has a finite number of overlay planes, and certain conditions—such as complex CSS transforms, rotations, or non-rectangular clipping applied to the video element—cannot be handled by the display controller's fixed-function hardware.

Firefox manages this through two mechanisms:

  • Browser-Level Fallback: If WebRender detects that a webpage applies complex shaders or unsupported CSS properties to a video, it will not place the video into an independent DirectComposition visual. Instead, it composites the video into the main browser SwapChain using the GPU's 3D shaders.
  • OS-Level Fallback: If Firefox sends a valid video SwapChain to the DWM, but the system runs out of hardware overlay planes (for example, if another application is already using them), DirectComposition handles the fallback automatically. The DWM simply uses the GPU's 3D engine to compose the Firefox visual tree into the desktop framebuffer. This ensures smooth playback continues without requiring Firefox to tear down and rebuild its rendering pipeline.