Pitch Accent vs Lexical Stress in TTS Models

Synthesizing natural-sounding speech requires accurate prosodic control, yet languages mark word-level prominence in fundamentally different ways. While languages like English rely on lexical stress—a complex interplay of pitch, duration, amplitude, and vowel quality—Japanese utilizes a pitch accent system governed almost exclusively by discrete fundamental frequency (\(F_0\)) transitions across timing units called morae. For modern neural Text-to-Speech (TTS) systems, this difference fundamentally shifts how front-end text processors annotate linguistic features, how acoustic models predict pitch contours, and how severely synthetic errors impact listener comprehension.

Multidimensional Realization vs. Pure \(F_0\) Contours

The primary acoustic difference lies in the dimensions of speech that a TTS model must alter to produce correct prominence.

Lexical stress in stress-accent languages (such as English, Russian, or Spanish) is multidimensional. When a model synthesizes a stressed syllable, it must jointly elevate fundamental frequency (\(F_0\)), prolong duration, increase intensity, and retain full vowel quality. Conversely, unstressed syllables often undergo vowel reduction (e.g., centralization to a schwa). Consequently, acoustic models like FastSpeech 2 or VITS can distribute the perception of stress across duration, energy, and pitch variance adaptors simultaneously.

Japanese pitch accent, by contrast, is primarily unidimensional. It relies almost entirely on relative \(F_0\) register shifts—specifically a steep fall in pitch immediately following an accented mora (the accent kernel). Japanese vowels do not undergo qualitative reduction when unaccented, and mora durations remain remarkably uniform due to the mora-timed nature of the language. In TTS synthesis, pitch accent is an explicit contour problem: the model does not need to heavily modulate energy or vowel timbre to convey accent, but it must generate a precise, rapid downstep in the \(F_0\) trajectory.

Front-End Processing and Feature Encoding

Because the mechanisms differ, the front-end text analysis pipeline must supply distinct linguistic tokens to the acoustic synthesizer.

In English TTS, grapheme-to-phoneme (G2P) converters append discrete stress markers to vowels (e.g., primary stress /ˈ/, secondary stress /ˌ/, or no stress). Modern end-to-end models can often infer these directly from text or phoneme embeddings, relying on contextual word embeddings (such as BERT) to disambiguate homographs like "record" (noun) versus "record" (verb).

Japanese TTS requires far more rigorous front-end prosodic parsing. Standard Japanese (Tokyo dialect) operates on binary high (H) and low (L) pitch states across a sequence of morae. A Japanese TTS front-end (such as Open JTalk or MeCab paired with UniDic) must determine:

1. Morpheme boundaries and parts of speech.
2. Lexical accent kernels (which mora, if any, drops in pitch).
3. Accent sandhi and phrasing rules, where compounding words merge or shift accent positions.
4. Catathesis (downstep), where subsequent pitch peaks within the same prosodic phrase are attenuated.

These are fed to the model not merely as phonemes, but as structured full-context labels or explicit tone sequences (e.g., marking whether each mora is high or low, and the exact location of the pitch fall).

Model Sensitivity to Acoustic Errors

The functional role of pitch accent makes TTS models far less tolerant of \(F_0\) inaccuracies in Japanese than in stress-based languages.

In English, if a TTS model predicts a slightly unnatural pitch trajectory over a stressed syllable, listeners can still identify the stress via duration and vowel clarity. Minor prosodic deviations may sound robotic or foreign, but intelligibility is rarely compromised.

In Japanese, pitch accent is phonemic and serves to distinguish extensive sets of minimal pairs. For example:

• Ame with a fall after the first mora (HL) means "rain."
• Ame with a low-to-high pattern (LH) means "candy."
• Hashi can mean "chopsticks" (HL), "bridge" (LH with an unaccented particle), or "edge" (LH with an accented particle).

If a neural network flattens the \(F_0\) contour, introduces an inappropriate rise, or misplaces the timing of the pitch drop by even a single mora, it creates an immediate semantic error or strong cognitive dissonance for native speakers. As a result, Japanese TTS systems often require dedicated pitch loss functions, explicit \(F_0\) conditioning, or auxiliary prosody prediction networks to prevent the acoustic model from averaging out the sharp pitch falls characteristic of natural Japanese speech.