DeepONet: Learning Nonlinear Operators in Scientific AI

This article provides a comprehensive overview of Deep Operator Networks (DeepONet) and how they revolutionize scientific artificial intelligence. It explores the theoretical foundation of learning nonlinear continuous operators across infinite-dimensional function spaces. Specifically, it details the dual-network architecture—comprising the branch and trunk networks—and explains how their synergy enables the rapid, accurate approximation of complex physical systems and partial differential equations (PDEs).

The Shift from Function to Operator Learning

Traditional neural networks excel at mapping finite-dimensional inputs to finite-dimensional outputs. However, scientific computing often requires mapping an entire input function (such as initial conditions, boundary conditions, or forcing terms) to an output function (such as the dynamic state of a physical system). In mathematics, a mapping between two infinite-dimensional function spaces is called an operator.

Standard neural networks struggle with this because they are tied to a specific discretization or grid. If the grid resolution changes, the network must be retrained. DeepONet overcomes this limitation by learning the underlying continuous operator itself, making it resolution-independent and vastly more efficient than traditional numerical solvers.

The Theoretical Foundation

DeepONet is built directly upon the Universal Approximation Theorem for Operators, introduced by Chen and Chen in 1995. This theorem states that a neural network with a single hidden layer can accurately approximate any nonlinear continuous operator.

To turn this mathematical theorem into a practical deep learning architecture, the creators of DeepONet structured the network to process two distinct types of information simultaneously: the input function and the exact coordinates where the output function is being evaluated. This division of labor is achieved through the branch and trunk networks.

The Branch Network: Encoding the Input Function

The branch network is responsible for processing the input function, which represents the variable parameters of the system (e.g., a heat source distribution or fluid viscosity profile).

Because computers cannot directly process continuous, infinite-dimensional functions, the input function must be represented discretely. DeepONet achieves this by evaluating the input function at a fixed number of locations, known as "sensors."

  1. Sensor Sampling: The input function \(u\) is measured at \(m\) fixed sensor locations, creating a finite vector of values.
  2. Feature Extraction: This vector is fed into the branch network. Depending on the structure of the sensor data, the branch network can be a Multi-Layer Perceptron (MLP), a Convolutional Neural Network (CNN) for grid-based data, or a Recurrent Neural Network (RNN) for sequential data.
  3. Latent Representation: The branch network outputs a \(p\)-dimensional vector of coefficients, which encapsulates the global features of the input function.

The Trunk Network: Encoding the Output Coordinates

The trunk network makes DeepONet truly continuous across the output domain. Instead of predicting the output function on a fixed grid, the trunk network takes the continuous coordinates \(y\) (such as spatial coordinates \(x, z\) and time \(t\)) where the user wants to evaluate the output function.

  1. Coordinate Input: The trunk network receives the specific domain coordinates \(y\) as its input.
  2. Basis Function Learning: Typically structured as a standard feed-forward neural network, the trunk network applies nonlinear activations to these coordinates.
  3. Output Vector: It outputs a \(p\)-dimensional vector (matching the dimension of the branch network's output). These outputs effectively act as a set of learned basis functions tailored to the specific physical system.

Combining Branch and Trunk to Learn the Operator

The magic of DeepONet happens when the outputs of the branch and trunk networks are merged. To evaluate the operator \(G(u)\) at a specific coordinate \(y\), DeepONet computes the dot product (inner product) of the branch output vector and the trunk output vector, often adding a final bias term.

Mathematically, this means the trunk network constructs the spatial-temporal basis functions of the system, while the branch network determines the correct weights to apply to those basis functions based on the specific input function.

Because the trunk network accepts any continuous coordinate \(y\), the resulting output can be evaluated anywhere in the domain without retraining. Once trained, DeepONet can predict the solutions to complex, nonlinear PDEs in a fraction of a second—orders of magnitude faster than traditional finite element or finite volume methods.