How Singing Synthesis Handles Vowel Prolongation

Singing voice synthesis systems sustain prolonged vowels naturally by replacing static sample-looping with dynamic acoustic modeling. While conventional text-to-speech relies on basic time-stretching that causes metallic buzzing and phase artifacts, singing synthesizers decouple voice source characteristics from vocal tract resonances. By leveraging spectral decomposition, continuous trajectory prediction for pitch and dynamics, and neural vocoding, modern engines render extended notes that retain the organic micro-variations, breathiness, and vibrato characteristic of real human performance.

The Problem with Basic Text-to-Speech Looping

Basic text-to-speech (TTS) engines typically handle sustained phonemes using techniques like Pitch-Synchronous Overlap and Add (PSOLA) or direct audio frame repetition. In spoken language, vowels are rarely held longer than a few hundred milliseconds, making simple frame duplication acceptable for short durations.

In singing, however, a single vowel can easily span several seconds. Repeating identical audio frames produces a perfectly periodic waveform that human ears perceive as an unnatural, electronic "buzz." This repetition eliminates the natural phase shifts, turbulence, and acoustic drift found in real vocal tracts, flattening the voice into an artificial drone.

Decoupling Harmonics and Aperiodicity

To avoid repetitive loops, modern singing synthesizers analyze and synthesize audio using vocoders like WORLD or STRAIGHT. These systems decompose the vocal signal into three independent streams:

1. Fundamental Frequency (\(F_0\)): The underlying pitch trajectory.
2. Spectral Envelope: The frequency response shaped by the throat, mouth, and tongue (formants).
3. Aperiodic Energy: The unvoiced breath, aspiration, and turbulence within the vocal tract.

By isolating these parameters, the synthesizer avoids duplicating raw audio waveforms. Instead, when an engine prolongs a vowel, it stretches these abstract mathematical parameters across the musical duration. The noise component remains stochastic (randomly generated) rather than repetitive, preventing the metallic phase cancellation that occurs when static noise frames are overlaid.

Dynamic Trajectory Modeling and Vibrato

A sustained human vocal note is never static. Natural singers continuously introduce pitch drift, formant modulation, and vibrato. Singing voice synthesis (SVS) systems explicitly model these temporal contours.

• Frequency Modulation: Synthesizers map target musical notes into continuous \(F_0\) curves rather than flat steps. Vibrato is synthesized via low-frequency oscillators or learned statistical models that fluctuate pitch between 4 to 7 Hz, varying depth and rate dynamically across the vowel's duration.
• Formant Transitions: As lungs empty and vocal fold tension shifts, the shape of the vocal tract undergoes subtle adjustments. Synthesizers model slow, continuous movements in the spectral envelope to simulate these shifts.
• Energy and Tremolo: Amplitude is modulated along an expressive curve, generating a dynamic swell or decay that gives the note musical direction.

Neural Vocoding and Generative Acoustic Models

The transition to deep learning has fundamentally eliminated looping artifacts in contemporary singing engines such as Synthesizer V, CeVIO, and DiffSinger.

Modern architectures utilize diffusion models, generative adversarial networks (GANs), or autoregressive transformers trained on extensive vocal datasets:

• Diffusion and Autoregressive Predictors: Given a musical score with explicit phonetic durations, the acoustic model predicts intermediate representations (such as mel-spectrograms) frame-by-frame across the entire prolonged duration. Because the network considers the full temporal context, it synthesizes unique spectral frames from the vowel's onset to its decay, ensuring no two frames are identical.
• Neural Vocoders: Frameworks like HiFi-GAN convert the generated spectrograms into audio waveforms. Trained on complex loss functions that evaluate frequency and phase coherence, neural vocoders reconstruct high-frequency details, vocal fry, and breath noise in a continuous, organic stream, making vowel prolongation indistinguishable from an authentic human vocal performance.