How Self-Organizing Maps Preserve Topology on a 2D Grid

Self-Organizing Maps (SOMs), developed by Teuvo Kohonen in the 1980s, are unsupervised artificial intelligence neural networks designed for dimensionality reduction and data visualization. This article explains the mechanics of how SOMs project complex, high-dimensional data onto a simple two-dimensional grid while strictly preserving the topological neighborhood relations of the original data space. By examining competitive learning, the Best Matching Unit, and neighborhood functions, you will understand how these networks self-organize without human supervision.

The Architecture of a Self-Organizing Map

To understand how topology is preserved, it is first necessary to understand the structure of the network. A SOM consists of two layers: an input layer and an output layer (often called the competitive layer or map space).

The output layer is arranged as a two-dimensional grid of nodes (neurons). Unlike traditional neural networks that pass data through multiple hidden layers to predict a target, every node in the SOM's 2D grid is directly connected to the input data via a weight vector. This weight vector has the exact same number of dimensions as the input data. Therefore, each node has two positions: a fixed physical location on the 2D output grid, and a variable position in the high-dimensional input space defined by its weights.

Competitive Learning and the Best Matching Unit

SOMs do not use backpropagation or gradient descent because there are no target labels in unsupervised learning. Instead, they rely on competitive learning.

When a high-dimensional data point is fed into the network, the nodes on the 2D grid compete to see which one most closely resembles the input. The network calculates the Euclidean distance between the input vector and the weight vector of every single node on the grid. The node with the weight vector closest to the input vector is declared the winner. This winning node is called the Best Matching Unit (BMU).

The Neighborhood Function: The Key to Topology

If the network only updated the weights of the BMU, the result would be simple clustering (similar to k-means), and no topological relationships would be preserved. The preservation of topology happens because of the neighborhood function.

Inspired by biological brain functions—where neighboring neurons tend to process similar sensory information—Kohonen designed the SOM so that the BMU excites the nodes physically adjacent to it on the 2D grid. When the BMU is pulled closer to the input data in the high-dimensional space, it drags its 2D grid neighbors along with it.

The mathematical mechanism usually employed for this is the Gaussian neighborhood function. The degree to which a neighboring node's weights are updated depends on its physical distance from the BMU on the 2D grid.

• The BMU receives the maximum weight update, moving its weights significantly closer to the input data.
• Immediate neighbors on the 2D grid receive a strong update, moving them toward the same input data.
• Distant nodes on the 2D grid receive little to no update.

Iterative Convergence and the "Elastic Net" Effect

The preservation of topology is an iterative process that requires two parameters to decay over time: the learning rate and the neighborhood radius.

1. The Ordering Phase: Early in the training process, the learning rate is high, and the neighborhood radius is very large (often encompassing the entire 2D grid). When a BMU is found, a massive section of the grid is pulled toward the input data. This creates a global ordering of the map, unfolding the grid in the high-dimensional space.
2. The Convergence Phase: As training progresses over thousands of iterations, both the learning rate and the neighborhood radius shrink. Eventually, the radius becomes so small that only the BMU and its immediate adjacent neighbors are updated. This fine-tunes the map to the local features of the data.

You can visualize the 2D grid as a flexible, elastic net suspended inside the high-dimensional data space. Every time a data point is introduced, a node on the net is pulled toward that data point, stretching the net and pulling adjacent nodes along with it.

Because nodes that are physically close on the 2D grid are repeatedly forced to adapt to the same or similar inputs, their weight vectors become similar. Consequently, data points that are close to each other in the complex, high-dimensional space will consistently activate nodes that are physically close to each other on the flat, 2D grid. This is how Teuvo Kohonen’s Self-Organizing Maps successfully preserve topological neighborhood relations.