Preserving Voice Identity in Extreme TTS Screams

Synthesizing extreme emotional states like screaming and severe distress while retaining a speaker's unique vocal identity presents a fundamental challenge for real-time Text-to-Speech (TTS) systems. Under intense physical strain, human vocal physiology undergoes nonlinear changes—such as severe pitch shifts, vocal fold desynchronization, and chaotic noise—which typically cause standard speaker embeddings to collapse. To overcome these limitations, modern architectures leverage disentangled representation learning, extreme prosody extrapolation, dedicated neural vocoder adaptations, and cross-speaker transfer techniques to deliver believable, identity-consistent distressed speech with low latency.

Disentangled Latent Representation Learning

The primary failure point during scream synthesis is the entanglement of speaker identity (timbre) with acoustic prosody (pitch, energy, and strain). When a model increases energy to simulate a scream, it frequently drifts toward the average voice found in the screaming training data, losing the target speaker's unique characteristics.

• Metric Learning for Robust Encoders: Standard speaker encoders trained on neutral speech break down at extreme fundamental frequencies (\(F_0\)). Utilizing loss functions like Generalized End-to-End (GE2E) or Sub-center ArcFace on multi-condition datasets forces the encoder to map the speaker's identity to a stable manifold regardless of vocal strain.
• Adversarial Decoupling: Training the acoustic model with a gradient reversal layer against an emotion classifier ensures that the speaker embedding vector contains zero prosodic or emotional leakage. This isolation allows the prosody predictor to scale pitch and decibels independently without warping identity characteristics.

Robust Pitch Modeling and Subharmonic Synthesis

Extreme distress pushes vocal folds into non-periodic, chaotic regimes characterized by biphonation, subharmonics, and vocal fry. Standard linear pitch modeling cannot capture these phenomena.

• Continuous Pitch Representations: Traditional discrete pitch contours fail at scream thresholds. Real-time models benefit from continuous wavelet transforms (CWT) or continuous pitch representations that smoothly extrapolate into high registers without generating robotic artifacts.
• Aperiodic and Roughness Injection: Real-time TTS systems can modulate the harmonic-to-noise ratio (HNR) dynamically. By conditioning the acoustic model with an explicit aperiodicity map, the system introduces high-frequency turbulence and vocal fry directly into the acoustic frames, mimicking physiological strain while keeping the underlying formant ratios of the speaker intact.

Fast-Inference Vocoders for Chaotic Waveforms

Acoustic models typically output mel-spectrograms, but the neural vocoder is responsible for turning those features into raw audio. Screams feature dense, saturated frequency bands that standard lightweight vocoders smudge.

• Anti-Aliased Periodic Discriminators: In low-latency Generative Adversarial Network (GAN) vocoders—such as variants of HiFi-GAN or BigVGAN—incorporating snake activations and anti-aliasing layers allows the vocoder to synthesize high-frequency harmonics without phase distortion or metallic ringing.
• Source-Filter Neural Vocoders: Decoupling the glottal source from vocal tract filter modeling allows the vocoder to inject raw chaotic noise into the excitation signal while forcing the filter to adhere strictly to the target speaker's established vocal tract resonances (formants).

Cross-Speaker Emotion Transfer and Data Adaptation

High-quality screaming audio from a specific target speaker is rarely available in large quantities. Real-time TTS systems rely on transfer methods to synthesize distress for speakers whose training data is entirely neutral.

• Latent Space Deformation: Acoustic models trained on expressive, multi-speaker corpuses can learn an "extreme distress" direction vector in the latent prosody space. Applying this vector to a neutral speaker's latent representation shifts their vocal mode into a scream while clamping their identity embedding to prevent identity drift.
• Reference-Based Prosody Encoders: During runtime, a low-latency reference encoder extracts the prosodic envelope of an actual recorded scream (from an actor or donor) and transfers the expressive contours to the target speaker’s model, preserving the target's timbre over the donor's performance dynamics.

Latency Optimization for Real-Time Execution

Preserving identity under extreme vocal conditions requires complex conditioning, which can increase computational overhead. To maintain real-time streaming capability:

• Non-Autoregressive Feed-Forward Backbones: Fully parallel systems (such as non-autoregressive flow-matching or modified FastSpeech-style architectures) process full or chunked phrases instantaneously, avoiding the high latency and compounding error rates of autoregressive models during chaotic generation.
• Feature-Wise Linear Modulation (FiLM): Instead of compute-heavy multi-head cross-attention across all layers, injecting the speaker identity embedding using simple affine transformations (FiLM) preserves low inference latency while maintaining robust conditioning throughout deep generative layers.