How Does Processing-in-Memory Embed Logic in DRAM?

Processing-in-Memory architecture addresses the memory bus bottleneck in classical computing by placing computational logic directly inside or adjacent to DRAM arrays. Instead of transferring massive datasets back and forth between main memory and the processor, PIM executes operations where the data already resides. This approach reduces data movement, lowers energy consumption, and increases effective memory bandwidth for data-intensive workloads.

The Memory Bus Bottleneck in Classical Computing

In traditional von Neumann architectures, the central processing unit and main memory are physically separated and connected by a shared memory bus. Every arithmetic or logical operation requires fetching operands from DRAM across this bus, processing them in CPU registers, and writing results back.

Because processor speeds have scaled much faster than memory access speeds, the bus has become a severe performance limiter. Data movement often consumes more time and energy than the computation itself. If a workload processes \(N\) bytes of data with low arithmetic intensity, the total execution time \(T\) is dominated by memory transfer latency:

\[T \approx \frac{N}{B_{\text{bus}}}\]

where \(B_{\text{bus}}\) represents the peak bandwidth of the external memory channel.

DRAM Array Organization

To understand how PIM embeds logic, it is necessary to look at how a DRAM chip is structured. A DRAM die is divided into multiple banks, and each bank consists of two-dimensional subarrays of memory cells.

Each memory cell stores a single bit as an electrical charge in a capacitor, accessed through an access transistor. Cells in the same row share a wordline, while cells in the same column share a bitline. At the edge of each subarray sits a row buffer composed of sense amplifiers, which detect and amplify the small voltage changes on the bitlines when a row is activated.

Embedding Logic Inside DRAM Arrays

PIM architectures embed computation at different levels of the DRAM hierarchy, depending on the target workload and hardware constraints.

Some PIM designs perform bitwise operations directly within the DRAM subarray without adding complex digital logic. By simultaneously activating multiple wordlines in the same subarray, the charge stored in different cells shares across the bitline.

The sense amplifier then resolves the resulting voltage to a logic high or low. For example, activating three rows simultaneously can compute a bitwise majority function:

\[\text{MAJ}(A, B, C) = (A \land B) \lor (B \land C) \lor (A \land C)\]

By setting one of the rows to a fixed \(0\) or \(1\), the same mechanism performs bitwise AND or OR operations across an entire row of thousands of bits in parallel.

Digital Logic Near Sense Amplifiers and Banks

Other PIM implementations place lightweight digital processing units—such as arithmetic logic units (ALUs), vector units, or programmable cores—directly next to the row buffers or at the DRAM bank level.

Because these units sit behind the external I/O pins, they access data at the full internal bank bandwidth rather than the narrow external bus. Multiple banks can operate concurrently, multiplying aggregate throughput by the number of banks \(K\):

\[B_{\text{internal}} = K \times B_{\text{bank}}\]

3D-Stacked Memory Logic Layers

In 3D-stacked memory technologies such as High Bandwidth Memory (HBM), multiple DRAM dies are stacked vertically and connected using Through-Silicon Vias (TSVs). A dedicated logic die at the base of the stack houses processing elements. The dense vertical interconnects provide high-bandwidth, low-energy pathways between the memory layers and the logic layer.

Overcoming Classical Limitations

By keeping data inside the DRAM chip during computation, PIM eliminates unnecessary round-trip transfers over the memory bus. The CPU offloads data-parallel kernels—such as vector additions, matrix-vector multiplications, and database scans—to the memory-side logic and retrieves only the final aggregated results. This division of labor relieves pressure on the memory channel and improves both system throughput and energy efficiency.