How Chromium Shares Web Platform Tests With Other Browsers

Web Platform Tests (WPT) is a collaborative, cross-browser test suite designed to ensure that web specifications are implemented consistently across all major web engines. This article explains the automated two-way synchronization process that allows the Chromium project to share tests seamlessly with the central WPT repository, enabling other browser vendors like Mozilla (Firefox) and Apple (Safari) to run, update, and contribute to the exact same tests.

The Central Hub: The WPT GitHub Repository

At the core of the sharing mechanism is the central WPT repository hosted on GitHub. Rather than browser vendors sending code directly to one another, they all interact with this single, vendor-neutral repository. It acts as the ultimate source of truth for web standards testing.

Chromium's Local Integration

Chromium does not run tests directly from the external GitHub repository during its daily development. Instead, it maintains a copy of the WPT suite inside its own massive source tree (specifically within the Blink rendering engine directory). This setup allows Chromium developers to write browser features and WPT tests in the exact same commit using Chromium's native code review tool, Gerrit.

Two-Way Automated Synchronization

To keep its local copy aligned with the central GitHub repository, Chromium relies on two automated bots that handle exporting and importing continuously.

1. Exporting Tests from Chromium

When a Chromium engineer writes a new test or modifies an existing one in the local WPT directory, the WPT Exporter bot detects the change.

  • As soon as the engineer uploads the code change for review in Chromium, the bot automatically creates a corresponding Pull Request (PR) in the central WPT GitHub repository.
  • Once the change passes review and is merged into Chromium's codebase, the bot automatically merges the PR on GitHub.
  • This makes the Chromium test immediately available to Firefox, Safari, and other WPT consumers.

2. Importing Tests into Chromium

Conversely, the WPT Importer bot continuously fetches the latest changes from the central WPT GitHub repository and copies them into Chromium's codebase.

  • The bot pulls commits that were recently added to GitHub by Mozilla, Apple, or independent specification authors.
  • It runs the newly imported tests against the Chromium browser.
  • If any imported tests fail in Chromium, the bot automatically updates Chromium's local "expectation files" (which mark the test as expected to fail) so the new tests can be merged without breaking Chromium's automated build pipeline.
  • Finally, the bot files bug reports for Chromium engineers to investigate the new failures.

How Firefox and Safari Participate

Mozilla and Apple use similar workflows to connect their internal repositories to the central WPT hub, completing the sharing circle.

  • Firefox (Mozilla): Mozilla maintains a copy of WPT inside its mozilla-central repository. Just like Chromium, Mozilla uses an automated two-way synchronization system that creates GitHub PRs when Firefox developers write tests, and regularly imports upstream WPT changes (including those from Chromium) back into Firefox.
  • Safari (Apple): Apple integrates WPT directly into the WebKit repository. WebKit engineers use dedicated tooling to import tests from the central WPT repository and export their local WPT modifications back to GitHub.

Managing Browser Differences

Because browsers implement web features at different paces, a shared test might pass in Chromium, fail in Safari, and time out in Firefox. To share the same test files without breaking each other's continuous integration systems, vendors only share the test logic, not the test results.

Each browser vendor maintains its own metadata and expectation files locally within its own repository. When Chromium imports a new test written by a Firefox developer for a feature Chromium hasn't built yet, Chromium accepts the test file as-is, but logs a local exception acknowledging that the test will fail until Chromium builds the feature. This strict separation of shared test logic and local test expectations is what allows competing browser engines to share a single testing codebase effortlessly.