Bypassing AI Guardrails: Crescendo and Typography Attacks

This article explores the mechanics behind two sophisticated adversarial techniques—multi-turn crescendo jailbreaks and visual typography attacks—and explains how they successfully bypass safety classifiers in multimodal artificial intelligence models. By examining the architectural vulnerabilities of large language and vision-language models, this overview details how attackers manipulate conversational context and visual processing pathways to evade modern AI guardrails.

Understanding Multimodal Safety Classifiers

Before examining the attacks, it is necessary to understand how multimodal AI models enforce safety. Most advanced models utilize a dual-layered defense mechanism:

Input/Output Guardrails: External classifiers that scan text and images for prohibited content before the main model processes the input, and again before the output is shown to the user.
Intrinsic Alignment: Training techniques, such as Reinforcement Learning from Human Feedback (RLHF), that condition the core model to refuse harmful requests naturally.

Both crescendo and visual typography attacks succeed by exploiting the blind spots between these two layers, specifically targeting how models process context over time and how they unify different data modalities (text and vision) into a single mathematical space.

Multi-Turn Crescendo Jailbreaks

A multi-turn crescendo jailbreak is a conversational attack that manipulates a model’s context window gradually. Instead of issuing a single, explicitly harmful prompt that would immediately trigger an input classifier, the attacker uses a sequence of benign prompts to slowly guide the model toward a prohibited topic.

How Crescendo Attacks Bypass Defenses:

Evasion of Stateless Classifiers: Many external safety classifiers evaluate prompts in isolation or only look at a limited subset of the conversation history to save computational resources. Because each individual prompt in a crescendo attack appears harmless, the external classifiers fail to flag the interaction.
Exploiting In-Context Learning: Large language models are designed to be helpful and maintain coherence based on prior turns in the conversation. As the attacker nudges the model closer to the restricted topic, the model's own previous responses become part of the context.
Attention Dilution: In transformer-based architectures, the self-attention mechanism weighs the importance of different tokens in the context window. By filling the context window with a lengthy, seemingly educational or hypothetical narrative, the model's intrinsic safety alignment is overridden by its stronger training objective: to complete patterns and remain consistent with the established conversational trajectory.
Refusal Suppression: Once the model has generated content adjacent to the restricted topic in Turn 3 or 4, asking it to take one final step in Turn 5 does not register as a severe violation of its alignment, because the semantic leap from the immediate context to the harmful output is minimal.

Visual Typography Attacks

Visual typography attacks (often considered a form of indirect prompt injection or cross-modal jailbreaking) target Vision-Language Models (VLMs). In this attack, the adversary takes a prohibited text command, renders it as an image (simply typing the text onto a blank background), and feeds the image to the model alongside a benign text prompt, such as "Follow the instructions in this image."

How Typography Attacks Bypass Defenses:

Circumventing Text-Only Input Filters: The primary vulnerability lies in the separation of modalities during pre-processing. Text-based safety classifiers scan the user's text prompt. Because the text prompt is merely "Follow the instructions in this image," the text filter registers no threat.
Inadequacy of Image Classifiers: Concurrently, the image is sent to a visual safety classifier. Traditional image classifiers are trained to detect explicit visual harm (such as violence, gore, or adult content). They are generally not equipped with Optical Character Recognition (OCR) paired with semantic intent analysis. To the image classifier, an image of black letters on a white background looks completely safe.
Late-Stage Modality Fusion: Once the inputs pass the initial filters, the VLM's vision encoder converts the image into visual tokens, and the text encoder converts the text into text tokens. Both sets of tokens are projected into a shared embedding space where the core model processes them together. The core model possesses advanced OCR capabilities and reads the typography within the image directly inside this shared space.
Alignment Discrepancy: Most safety alignment (RLHF) is heavily skewed toward text inputs. When a harmful instruction arrives via visual tokens rather than text tokens, the model’s internal refusal mechanisms are significantly less likely to activate. The model successfully reads the text inside the image and executes the command because its safety training was not robustly generalized to the visual embedding space.

The Convergence of Vulnerabilities

Both techniques highlight a fundamental flaw in current AI safety architectures: the disconnect between perception and comprehension.

In a crescendo attack, the safety classifiers perceive individual trees, while the core model comprehends—and gets lost in—the forest of the broader context. In a visual typography attack, the safety classifiers perceive pixels and harmless text separately, while the core model comprehends the unified semantic meaning of the text rendered within those pixels. Addressing these vulnerabilities requires shifting from static, modality-specific filters to dynamic, context-aware, and natively multimodal safety evaluations.