One-Class SVM: Hyperspheres for Novelty Detection in AI

This article provides a concise explanation of how a One-Class Support Vector Machine (OC-SVM), specifically through Support Vector Data Description (SVDD), detects anomalies in artificial intelligence. It covers how the algorithm maps normal training data into a high-dimensional space, calculates a minimal bounding hypersphere to encapsulate this data, utilizes slack variables to handle natural outliers, and identifies new, unseen data points as novelties if they fall outside the learned boundary.

Mapping Data to a Higher Dimension

Real-world data is rarely shaped like a clean sphere. To wrap a sphere around complex data, the One-Class SVM first projects the normal training data into a higher-dimensional feature space.

It achieves this using a mathematical function known as a kernel, most commonly the Radial Basis Function (RBF) kernel. The kernel trick calculates the relationships between data points as if they were in a higher dimension without actually having to compute the complex coordinates. In this transformed space, the normal data points naturally cluster together, making it possible to enclose them within a spherical boundary.

Defining the Minimal Bounding Hypersphere

Once the data is mapped, the algorithm seeks to find the smallest possible hypersphere that encompasses the normal data cluster. A hypersphere is simply the high-dimensional equivalent of a 2D circle or a 3D sphere.

The geometry of this hypersphere is defined by two parameters:

  • Center (\(a\)): The central point of the sphere in the high-dimensional space.
  • Radius (\(R\)): The distance from the center to the boundary of the sphere.

The mathematical objective of the algorithm is to minimize the volume of the sphere by minimizing \(R^2\). By forcing the radius to be as small as possible, the boundary wraps tightly around the normal training data, reducing the chance of misclassifying abnormal data as normal.

Managing Outliers with Slack Variables

If the algorithm strictly forced every single training point inside the sphere, a single extreme data point would drastically inflate the radius, ruining the tight fit. To prevent this, the algorithm introduces slack variables (\(\xi\)) and a hyperparameter often denoted as \(C\) (or \(\nu\) in related formulations).

Slack variables allow certain training data points to fall outside the hypersphere, treating them as anomalies within the training set itself. However, the algorithm applies a penalty for every point left outside. The optimization problem thus becomes a balancing act: minimize the radius \(R^2\) while simultaneously minimizing the penalties of the slack variables.

The data points that end up sitting directly on the boundary of the finalized hypersphere (or outside it) are the Support Vectors. These specific points completely dictate the final position and size of the sphere; all the data points safely inside the sphere can be ignored once training is complete.

Detecting Novelties in New Data

After the optimal center (\(a\)) and radius (\(R\)) are calculated using the support vectors, the model is ready for novelty detection.

When a new, unseen data point (\(z\)) is introduced to the AI model, the system maps it into the same high-dimensional space and calculates its distance from the center of the hypersphere:

  1. Normal Data: If the distance from the new point to the center \(a\) is less than or equal to the radius \(R\), the point falls inside the hypersphere and is classified as normal.
  2. Novelty/Anomaly: If the distance from the new point to the center \(a\) is greater than the radius \(R\), the point falls outside the hypersphere. The algorithm flags this point as a novelty or anomaly.