How Chromium’s Get Image Descriptions Feature Works

The "Get Image Descriptions" feature in Chromium-based browsers uses cloud-based machine learning to automatically generate alternative text (alt text) for unlabeled graphics, making the web more accessible for blind and low-vision users. This article explains the step-by-step process of how Chromium detects missing labels, processes visual data through external servers, and delivers accurate descriptions directly to screen readers.

1. Detecting the Need for a Description

The process begins only when Chromium detects that a user has an active screen reader running, such as NVDA, JAWS, or VoiceOver. As the user navigates a webpage, the browser analyzes the page's code. It looks specifically for images that lack an alt attribute, have an empty alt attribute where one is clearly needed, or contain unhelpful placeholder text like "image.jpg" or "DSC_001.png". To avoid clutter, the browser ignores tiny graphics, tracking pixels, and purely decorative elements.

2. Transmitting the Image

Once Chromium identifies an unlabeled, meaningful image—and provided the user has opted into the "Get Image Descriptions" feature—the browser captures the image data. It then securely transmits the image to Google’s cloud servers for analysis. For privacy reasons, the browser only sends the image itself, stripping away the surrounding webpage context, user credentials, and browsing history. The image is deleted from the server immediately after the description is generated.

3. Analyzing the Graphic with Machine Learning

On the server, the image is routed through several advanced computer vision and machine learning models simultaneously:

  • Optical Character Recognition (OCR): This model scans the graphic for any embedded text, such as words on a street sign, a scanned document, or text within an infographic.
  • Object and Scene Recognition: This model identifies the primary subjects, background elements, and actions taking place within the image, such as "a person riding a bicycle on a mountain trail."
  • Logo and Landmark Detection: Specialized models check the image against databases of known corporate logos and famous geographical landmarks.

4. Synthesizing the Alt Text

After the machine learning models finish their analysis, an algorithm evaluates the confidence score of each result. If the AI is not confident in what it sees, it will discard the result to avoid giving the blind user false information.

If the confidence threshold is met, the system compiles the data into a concise, human-readable sentence. To ensure the user knows the description was generated by a computer rather than the website's author, the system prepends phrases like "Appears to be..." or "May be an image of..." to the final text. If the OCR model detects text, it will format the output as "Appears to say:" followed by the extracted words.

5. Updating the Accessibility Tree

The server sends the finalized text string back to the Chromium browser. Instead of altering the webpage's actual HTML source code, Chromium injects this new description directly into the browser's Accessibility Tree—the specialized structure that assistive technologies use to interpret a webpage. The user's screen reader immediately detects this update and reads the newly generated alt text aloud, allowing the user to understand the graphic seamlessly.