G2P Alignment Failures in Character-Based TTS

End-to-end character-based Text-to-Speech (TTS) models synthesize speech directly from raw text characters without requiring an external grapheme-to-phoneme (G2P) front-end. While this simplifies the pipeline, it shifts the burden of phonetic transcription, stress assignment, and temporal alignment onto internal neural mechanisms. This article examines the primary failure modes of implicit G2P alignment in character-based TTS systems, detailing why attention and duration mechanisms break down and how these errors manifest in synthesized speech.

Attention Drift and Alignment Collapse

In models relying on soft attention mechanisms (such as standard cross-attention in autoregressive decoders), alignment failure is the most prevalent issue. Because characters do not have a strictly linear or one-to-one relationship with acoustic frames, attention weights can disperse across non-adjacent characters.

This causes two common failure modes:

• Token Skipping: The attention mechanism jumps ahead over complex or unstressed graphemes, omitting syllables or whole words.
• Babbling and Looping: The decoder gets stuck attending to the same grapheme sequence repeatedly, resulting in infinite loops or stuttering audio.

While monotonic attention constraints and dynamic programming algorithms (like Monotonic Alignment Search) help prevent these issues, models can still fail when encountering unexpected punctuation or irregular character sequences.

Silent Letters and Orthographic Asymmetry

In languages like English or French, the orthography is deep and non-phonemic. Characters frequently have zero acoustic duration (e.g., the "k" and "gh" in "knight" or the "b" in "doubt").

Character-based alignment mechanisms struggle to assign appropriate alignments to silent characters:

• Ghost Sounds: The model attempts to assign a non-zero duration to a silent grapheme, leading to unnatural pauses, glottal stops, or faint, hallucinated consonant releases.
• Smearing: The acoustic characteristics of neighboring sounds are incorrectly smeared over the silent character, degrading clarity and causing muffled articulation.

Homograph and Polyphone Mispronunciation

End-to-end character-based systems must deduce pronunciation entirely from context. When encountering heteronyms (words with identical spelling but different pronunciations based on part of speech or semantics), implicit G2P mapping frequently fails.

Common examples include:

• "read" (/ɹiːd/ vs. /ɹɛd/)
• "lead" (/liːd/ vs. /lɛd/)
• "wind" (/wɪnd/ vs. /waɪnd/)

Without an explicit linguistic dictionary or deep semantic understanding, character-level encoders often fall back on the statistical majority class found in the training corpus. When forced to align a contextually incorrect phoneme target, the alignment module creates awkward pitch transitions and distorted vowel durations.

Out-of-Vocabulary (OOV) Names and Loanwords

Character-based models are theoretically capable of pronouncing any arbitrary string of characters. However, implicit G2P alignment relies heavily on common n-gram orthographic patterns observed during training.

When presented with:

• Non-native proper nouns (e.g., "Tchaikovsky")
• Borrowed words with foreign spelling conventions (e.g., "rendezvous")
• Unconventional domain-specific abbreviations or neologisms

The network tries to force regular phonetic rules onto irregular spelling. This causes alignment mismatches where the duration predictor miscalculates the required frames, leading to rushed, compressed, or heavily garbled segments of speech.

Acoustic-Phonetic Entanglement

In explicit pipelines, text-to-phoneme conversion and phoneme-to-acoustic modeling are decoupled. In character-based models, G2P conversion and acoustic synthesis occur jointly.

Because the model must simultaneously determine what to say (phonemes) and how to say it (timbre, prosody, and duration), errors compound:

• Prosodic Bleed: Phonetic uncertainty translates directly into acoustic instability, producing flat or sudden pitch anomalies.
• Co-articulation Failures: The network fails to model transitions between characters that produce complex phonetic blends (such as "th" or "ch"), rendering them as distinct, awkwardly aligned individual sounds.