Circuit Breakers in AI: Routing Harm to Orthogonal States

This article explores how the Circuit Breakers alignment technique prevents artificial intelligence from generating harmful content. Instead of relying on fragile output filters or traditional refusal training, this method directly intervenes in the model's internal neural pathways. By detecting malicious representations during processing and routing them toward orthogonal refusal states, Circuit Breakers effectively neutralize threats mid-generation while preserving the AI's general capabilities.

Traditional AI safety methods, such as Reinforcement Learning from Human Feedback (RLHF), focus on training models to recognize harmful prompts and output a standard refusal. However, these defenses are vulnerable to adversarial attacks and complex jailbreaks. Attackers can bypass input filters by disguising their intent, forcing the model into a state where it feels compelled to complete the harmful request.

Circuit Breakers shift the focus from the model's inputs and outputs to its internal representations. As a large language model processes information, concepts are represented as multi-dimensional vectors within its hidden layers, known as the activation space. When a model prepares to generate dangerous content, specific patterns of neural activations—or harmful states—emerge inside this space, regardless of how clever or obfuscated the user's prompt was.

To stop this, researchers use a technique called Representation Engineering (RepE) to map out the exact regions of the activation space associated with harmful generation. Once these harmful internal states are identified, the Circuit Breaker mechanism is trained into the model, often using lightweight adapters like Low-Rank Adaptation (LoRA). This mechanism constantly monitors the AI's internal state as it generates text token by token.

When the model's internal trajectory begins to align with a known harmful representation, the Circuit Breaker trips. It interrupts the generation process by actively rerouting the model's internal vectors toward an "orthogonal refusal state."

In geometry and linear algebra, orthogonal vectors are perpendicular to one another, meaning they share a dot product of zero and have no overlapping components. In the context of an AI's activation space, an orthogonal state is a neural representation that is completely independent of the harmful state. By forcing the malicious internal representation to shift ninety degrees into an orthogonal direction, the model completely loses the semantic thread of the harmful concept it was about to generate.

Instead of landing in a random, incoherent space, the vector is specifically guided to a designated refusal state—or a total halt state—that is orthogonal to the harmful trajectory. Because the new state shares no informational overlap with the malicious state, the model cannot continue generating the dangerous material. It immediately defaults to a safe refusal or stops generating tokens altogether.

This geometric rerouting provides two major advantages. First, it makes the AI highly resilient against unseen jailbreaks. Even if a novel prompt tricks the outer layers of the model, the inevitable internal activation of the harmful concept will trip the breaker before the output is fully realized. Second, because the intervention targets only the specific vectors associated with harm and moves them to an isolated orthogonal space, it avoids degrading the model's helpfulness on benign topics, solving the common trade-off between AI safety and utility.