External Memory in Neural Turing Machines and DNCs

This article explores the mechanisms behind attaching external addressable memory matrices to artificial intelligence controllers, specifically focusing on Neural Turing Machines (NTMs) and Differentiable Neural Computers (DNCs). It provides a clear breakdown of the architecture connecting neural controllers to memory banks, the mathematical attention mechanisms used to read and write data, and the evolutionary leaps that allow DNCs to manage complex, dynamic memory allocation without overwriting critical historical data.

Traditional recurrent neural networks struggle to store and manipulate variables over long durations. NTMs and DNCs solve this by separating computation from memory storage. At the core of both architectures is a neural network controller—often a Long Short-Term Memory (LSTM) network or a feedforward network—acting as the central processing unit. This controller interacts with an external memory matrix, which serves as the system's RAM.

The memory matrix consists of \(N\) rows (memory locations) and \(M\) columns (vector size at each location). The controller does not access single, discrete memory locations like a traditional computer, because discrete selection is not differentiable and breaks backpropagation. Instead, the controller interacts with the memory using "blurry" read and write operations. It interacts with every memory location simultaneously, to varying degrees, using weightings defined by an attention mechanism.

To connect the controller to the memory matrix, the system uses "read heads" and "write heads." During each time step, the controller receives an external input and emits a set of interface parameters to these heads. These parameters generate normalized weighting vectors—vectors of values between 0 and 1 that sum to 1—which dictate where the heads should focus on the memory matrix.

To read from the memory matrix, a read head takes its generated weighting vector and computes a convex combination (a weighted sum) of all the row vectors in the memory matrix. If a specific location has a weight close to 1 and all others are near 0, the controller effectively reads just that single row.

Writing to the memory matrix is borrowing from the gating mechanisms of LSTMs and is split into two steps: an erase operation followed by an add operation. The controller emits a write weighting vector, an "erase vector," and an "add vector." First, the memory rows are multiplied by the inverse of the erase vector, scaled by the write weights. If both the weight and the erase vector at a specific location are 1, the existing data is completely zeroed out. Next, the add vector is multiplied by the write weights and added to the newly erased memory matrix, updating the stored information.

The true innovation in attaching this memory lies in how the attention weights are calculated—a process known as addressing. NTMs use a combination of two addressing mechanisms:

1. Content-Based Addressing: The controller emits a "key" vector, which is compared to every row in the memory matrix using a similarity measure, typically cosine similarity. This allows the controller to find data based on what the data looks like, effectively acting as an associative memory recall.
2. Location-Based Addressing: This allows the controller to iterate through memory sequentially, regardless of what the data looks like. Using interpolation gates, the controller can choose to keep its previous weighting, shift the weights up or down by rows using a convolutional shift mechanism, and sharpen the weighting distribution to prevent the focus from becoming too dispersed over time.

While NTMs successfully attach external memory, they suffer from a major limitation: contiguous memory blocks. Because NTMs use simple rotational shifts for location-based addressing, they often accidentally overwrite previously stored information when allocating new memory. Differentiable Neural Computers (DNCs) were created to fix this by upgrading the addressing mechanisms.

DNCs attach their memory matrix using three distinct addressing methodologies. Like the NTM, they use content-based lookup, but they replace the NTM's rudimentary location-based shifts with two advanced systems:

1. Dynamic Memory Allocation: A DNC tracks the "usage" of each memory row using a retention vector. When the controller writes data, it can ask for a completely unused location. The DNC calculates which rows have the lowest usage weights and directs the write head there. Furthermore, the read heads can dynamically free up memory locations once the data has been read and is no longer needed.
2. Temporal Memory Linkage: Because dynamic allocation scatters data randomly across the memory matrix to avoid collisions, a DNC cannot simply shift one row down to find the next piece of sequential data. To solve this, the DNC maintains a \(N \times N\) "temporal link matrix." Every time the controller writes to the memory, this link matrix updates to record the order in which rows were modified. This allows the read heads to traverse memory forward or backward in the exact chronological order the data was written, regardless of physical location in the matrix.

Because every operation—cosine similarity, weighted sums, vector multiplication, and link matrix updates—is entirely composed of continuous, linear, and differentiable math, the entire system remains end-to-end differentiable. As a result, standard gradient descent and backpropagation through time can train the AI controller not just on how to process data, but on how to physically organize, retrieve, and manage its own external memory.