How BYOL Trains AI Without Negative Sample Pairs

Bootstrap Your Own Latent (BYOL) is a self-supervised learning approach in artificial intelligence that learns visual representations without requiring negative sample pairs. Traditional contrastive learning methods need to compare an image with both similar (positive) and different (negative) images to prevent the AI from outputting the same meaningless result for every input. BYOL eliminates the need for negative samples by using an asymmetrical architecture composed of two neural networks: an online network and a target network. This article explains how BYOL processes image augmentations, updates its networks via an exponential moving average, and avoids representation collapse using this dual-network design.

To begin the training process, BYOL takes a single source image and applies two different random augmentations—such as cropping, color jittering, or flipping—to create two distinct views. One view is fed into the online network, and the other is fed into the target network. The goal is for the online network to predict the representation of the image generated by the target network, effectively bootstrapping its own latent representation to learn the core features of the image regardless of the augmentation applied.

The two networks have slightly different internal structures. The online network consists of three main components: an encoder (which extracts visual features), a projector (which maps features to a smaller latent space), and a predictor. The target network only contains an encoder and a projector, lacking the predictor component. The predictor in the online network is the critical piece that attempts to match the output of the target network's projector. The error between the online network's prediction and the target network's output is calculated using mean squared error.

A major challenge in training without negative samples is "representation collapse," where the network learns to cheat by outputting a constant, unchanging vector for every single image, immediately reducing the error to zero without learning anything useful. BYOL prevents this collapse through the specific way it updates the weights of the two networks. The online network is updated normally using standard backpropagation based on the calculated error. However, the target network never receives direct gradient updates from this error.

Instead of backpropagation, the target network is updated using an exponential moving average (EMA) of the online network's weights. After the online network learns and adjusts its parameters, the target network slowly shifts its own parameters to move slightly closer to the online network's new state. Because the target network changes very slowly, it provides a stable, gradually improving target for the online network to predict.

The combination of the extra predictor layer in the online network and the slow-moving EMA updates in the target network creates an asymmetry that makes representation collapse impossible. The target is constantly moving just enough to force the online network to learn meaningful visual features, rather than taking the shortcut of outputting a constant value. Once the training is complete, everything except the online network's encoder is discarded, leaving a powerful AI model capable of recognizing complex visual patterns.