AI in Spatial Transcriptomics: Mapping Genes to H&E Slides

Spatial transcriptomics revolutionizes modern pathology by merging molecular genetics with traditional tissue microscopy. Through the use of artificial intelligence, researchers can now directly overlay high-dimensional gene expression profiles onto standard Hematoxylin and Eosin (H&E) stained histology slides. This article provides an overview of how machine learning algorithms process histological images, align molecular data with cellular structures, enhance spatial resolution, and ultimately predict gene expression directly from tissue morphology.

Image Registration and Data Alignment

The first step in combining spatial transcriptomics with H&E slides is precise alignment, known as image registration. In a standard workflow, a tissue section is placed on a specialized array containing thousands of microscopic spots, each bearing unique spatial barcodes that capture mRNA from the tissue above it. Before the tissue is permeabilized to release this genetic material, a standard H&E image is captured.

Artificial intelligence, specifically computer vision algorithms, aligns the visual pixel coordinates of the H&E image with the physical coordinates of the sequenced barcode array. Deep learning models automatically correct for tissue warping, tearing, or shrinking that occurs during the chemical processing. This ensures that the sequencing data from a specific barcode maps to the exact corresponding physical location on the visual H&E slide.

Cell Segmentation and Deconvolution

Standard spatial transcriptomics capture spots are often larger than a single cell, meaning one spot might capture the genetic output of multiple cells at once. AI solves this resolution limit through cell segmentation and deconvolution.

Convolutional Neural Networks (CNNs) scan the H&E slide to identify individual cell nuclei and boundaries based on their staining intensity and shape. Once the AI knows the exact number and visual type of cells sitting inside a single capture spot, it uses statistical deconvolution algorithms. These algorithms take the mixed genetic soup captured at that spot and assign specific gene expression profiles to the individual cells identified by the computer vision model.

Predicting High-Dimensional Profiles from Morphology

Because spatial transcriptomics is expensive and covers limited areas of a tissue, AI is used to fill in the gaps and infer genetic data where sequencing did not physically occur. A cell's genetic activity directly dictates its physical appearance—such as the size of the nucleus, the texture of the cytoplasm, and how densely cells pack together.

Deep learning models, including Vision Transformers (ViTs) and Graph Neural Networks (GNNs), are trained on paired datasets of H&E images and spatial transcriptomics data. The AI breaks the H&E image into thousands of tiny patches and learns the complex visual patterns that correlate with the expression of thousands of specific genes.

Generating the Virtual Overlay

Once trained, the AI can perform "super-resolution" mapping. It takes a low-resolution grid of gene expression data and projects it onto the H&E slide at a continuous, single-cell resolution. Furthermore, these trained AI models can analyze a brand-new, standard H&E slide that has never undergone genetic sequencing and generate a virtual spatial transcriptomic overlay. By translating pixel patterns into molecular predictions, the AI creates a color-coded heatmap over the H&E tissue, showing pathologists precisely where specific genes are active within the cellular architecture.