Analog In-Memory Computing: The Physics of AI Acceleration
This article provides an overview of how analog in-memory computing chips overcome the bottlenecks of traditional processors by executing artificial intelligence workloads directly within memory arrays. It explores the architecture of crossbar arrays, the utilization of phase-change memory and memristors to store neural network weights, and how these components rely on fundamental physics—specifically Ohm's law and Kirchhoff's current law—to perform massive matrix-vector multiplications instantaneously.
The Von Neumann Bottleneck and In-Memory Computing
Traditional computing relies on the von Neumann architecture, which physically separates the central processing unit (CPU) from the memory unit. Modern artificial intelligence (AI) models, particularly deep neural networks, require millions or billions of parameters to be constantly shuttled back and forth between memory and the processor. This data movement consumes massive amounts of energy and limits processing speed, a limitation known as the von Neumann bottleneck.
Analog in-memory computing (AIMC) eliminates this bottleneck by performing calculations directly inside the memory where the data is stored. The primary mathematical operation in AI is matrix-vector multiplication (MVM). AIMC chips are specifically designed to execute MVM operations in a single computational step using the physical properties of memory devices arranged in a grid.
Crossbar Arrays: Memristors and Phase-Change Memory
The foundation of an analog in-memory computing chip is the crossbar array. A crossbar array consists of a grid of horizontal metal wires (wordlines) intersecting with vertical metal wires (bitlines). At every intersection, or crosspoint, a non-volatile resistive memory device connects the horizontal wire to the vertical wire.
Two primary technologies are used at these intersections:
- Memristors (Resistive Random-Access Memory): These are two-terminal electrical components that limit or regulate the flow of electrical current in a circuit and remember the amount of charge that has previously flowed through them. By applying specific voltage pulses, the internal atomic structure of the memristor changes, allowing it to be programmed to various states of electrical resistance.
- Phase-Change Memory (PCM): PCM devices use chalcogenide glass that can be switched between two physical states by applying heat via electrical pulses. The amorphous state has a disordered atomic structure and exhibits high electrical resistance. The crystalline state has an ordered atomic structure and exhibits low electrical resistance. By controlling the size of the crystalline and amorphous regions, the device can achieve a continuous spectrum of resistance levels.
In the context of AI, it is easier to work with conductance (the ease with which electric current flows), which is simply the inverse of resistance (\(G = 1/R\)). Before executing an AI task, the synaptic weights of a neural network's matrix are mapped and programmed into the crossbar array as specific conductance values within the memristors or PCM cells.
Multiplication via Ohm's Law
Once the AI model's matrix weights are stored as conductance values at the crosspoints of the array, the chip is ready to process input data (the vector). The input vector is converted from digital numbers into continuous analog voltages using Digital-to-Analog Converters (DACs).
These analog voltages are applied simultaneously to the horizontal rows of the crossbar array. As the voltage travels along a horizontal wire and passes through a memristor or PCM cell at an intersection, a fundamental physical law takes over: Ohm's law.
Ohm's law states that the electrical current (\(I\)) passing through a conductor is equal to the voltage (\(V\)) applied across it multiplied by its conductance (\(G\)), expressed mathematically as \(I = V \times G\).
In the crossbar array, the input data is the voltage (\(V\)), and the stored AI weight is the conductance (\(G\)). Simply by letting the electricity pass through the memory cell, the physical hardware naturally multiplies the two values together. The resulting electrical current (\(I\)) exiting that specific memory cell is the exact product of the input and the weight.
Accumulation via Kirchhoff's Current Law
Matrix-vector multiplication requires not just multiplying inputs by weights, but also summing those products together for each column. In a digital processor, this requires multiple clock cycles of fetching and adding numbers. In an analog crossbar array, this happens instantly via Kirchhoff’s current law.
Kirchhoff’s current law states that the total current entering a junction or a wire must equal the total current leaving it.
As each memory cell in a vertical column produces its own electrical current (via Ohm's law), those individual currents flow out of the cells and merge into the shared vertical bitline wire. Because currents naturally add together when they share a path, the total current reaching the bottom of a vertical column is the exact sum of all the currents generated by the memory cells in that column.
Mathematically, the total current at the bottom of the column represents \(\sum (V_i \times G_i)\), which is the complete dot product of the input vector and the column of matrix weights. Since a crossbar array has many columns, the chip calculates the dot products for the entire matrix simultaneously.
Completing the Loop
The final step of the process occurs at the bottom of the vertical columns. The accumulated analog currents are read by Analog-to-Digital Converters (ADCs). The ADCs translate the physical electrical currents back into digital binary numbers, which can then be passed to standard digital logic for activation functions or routed to the next crossbar array for the subsequent layer of the neural network. By relying on Ohm's law and Kirchhoff's law to do the heavy lifting, analog in-memory chips execute complex matrix math at the speed of electricity, drastically reducing the time and energy required for AI computation.