How Chromium Uses Machine Learning OCR for Accessible PDFs

This article explains how the Chromium web browser utilizes on-device machine learning and optical character recognition (OCR) to transform inaccessible, image-based PDFs into readable text for screen readers. It covers the initial detection of unselectable PDF content, the local execution of the machine learning model to extract text and layout data, and the final integration of this information into the browser's accessibility tree for assistive technologies.

1. Detecting the Inaccessible PDF

When a user opens a PDF in a Chromium-based browser while using a screen reader, the browser inspects the document's structure. Many PDFs are simply scanned images of physical paper and lack an underlying text layer. Chromium's PDF viewer (powered by the PDFium engine) detects when a page consists solely of an image and lacks the metadata required for a screen reader to announce the content.

2. Downloading the Local Machine Learning Model

To protect user privacy, Chromium does not send the PDF to a cloud server for processing. Instead, the browser triggers a built-in component called the Screen AI service. If the feature is being used for the first time, Chromium automatically downloads a lightweight, pre-trained machine learning OCR library directly to the user's device. This model runs entirely offline within a secure, sandboxed browser process.

3. Extracting and Processing the Image

Because a PDF can contain hundreds of pages, processing the entire document at once would consume too much memory and processing power. Chromium handles this by processing pages incrementally as the user scrolls or navigates through the document.

For each visible page, the browser performs the following actions:

Bitmap Generation: The PDF rendering engine converts the inaccessible PDF page into a standard bitmap image.
Preprocessing: The image is adjusted for contrast, orientation, and resolution to maximize the accuracy of the text recognition.
Text Detection: The machine learning model scans the image to locate bounding boxes where words, lines, and paragraphs exist.

4. Recognizing Characters and Layout

Once the text locations are identified, the OCR model analyzes the pixels within those boundaries to identify individual letters, numbers, and punctuation marks.

Beyond simple character recognition, the machine learning model also performs layout analysis. It evaluates the size, spacing, and positioning of the text blocks to infer the document's structure. This allows the model to determine the correct reading order, distinguish between columns, and identify structural elements like headings and paragraphs.

5. Building the Accessibility Tree

Screen readers do not read raw visual layouts; they rely on a structured data model built by the browser called the Accessibility Tree.

After the OCR model finishes extracting the text and its structural hierarchy, the Screen AI service translates this data into standard accessibility nodes. Each paragraph, heading, and line of text is assigned an appropriate role and stitched directly into the browser's existing accessibility tree for that specific PDF page.

Once the accessibility tree is updated with the newly generated OCR nodes, Chromium notifies the operating system's accessibility API. The user's screen reader (such as NVDA, JAWS, or VoiceOver) can then navigate the scanned PDF just like a standard webpage. The user can read the text line by line, jump between headings, and copy information, completely bypassing the limitation of the original image-only file.