Harmonic-Plus-Noise Model in Parametric TTS

The harmonic-plus-noise model (HNM) is a spectral decomposition framework used in parametric Text-to-Speech (TTS) engines to synthesize natural-sounding human speech. It operates by separating the speech signal into a deterministic (harmonic) part, which models the quasi-periodic vibrations of the vocal cords, and a stochastic (noise) part, which models turbulence and aspiration. In parametric TTS, this division allows the engine to accurately analyze, modify, and reconstruct both periodic voiced sounds (such as vowels) and aperiodic fricative sounds (such as /s/ or /sh/) with high acoustic fidelity.

The Core Architecture: The Maximum Voiced Frequency

The foundation of the HNM approach relies on dividing the frequency spectrum of a speech frame into two bands separated by a time-varying boundary called the Maximum Voiced Frequency (\(F_m\)).

• Frequencies below \(F_m\) are treated as predominantly harmonic (voiced).
• Frequencies above \(F_m\) are treated as purely stochastic (noise).

For fully voiced sounds, \(F_m\) can reach up to 4 kHz to 6 kHz, meaning most of the perceptible lower spectrum is modeled by harmonics. For unvoiced sounds, \(F_m\) drops to 0 Hz, causing the entire frame to be rendered as noise.

Representing Voiced Speech Components

Voiced speech occurs when airflow from the lungs causes the vocal folds to vibrate quasi-periodically. The HNM represents this deterministic structure through additive sinusoidal modeling:

1. Harmonic Sinusoids: Below the threshold \(F_m\), the signal is represented as a finite sum of harmonically related sinusoids. Each harmonic is defined by its frequency (an integer multiple of the fundamental frequency, \(F_0\)), a time-varying amplitude derived from the spectral envelope, and a phase value.
2. Phase Modeling: To maintain natural timbre and avoid an artificial, metallic sound, parametric TTS engines estimate or predict regular phase tracks (often combining a linear phase linked to pitch marks with a minimum-phase vocal tract filter component).
3. High-Frequency Aspiration: Above \(F_m\), even voiced vowels contain minor turbulence caused by air rushing past the vocal folds. The model represents this upper band using Gaussian white noise shaped by an envelope matching the vocal tract's spectral tilt.

Representing Fricative Speech Components

Fricatives are produced when airflow is forced through a narrow constriction in the vocal tract (such as the tongue against the teeth or palate), creating continuous turbulent noise rather than a periodic waveform.

1. Unvoiced Fricatives (e.g., /s/, /f/, /θ/):

   • In completely unvoiced fricatives, there is no vocal fold vibration, so \(F_0\) is absent and \(F_m\) is set to 0 Hz.
   • The harmonic component is completely disabled.
   • The stochastic component generates the sound by passing a random noise source (typically white Gaussian noise) through a time-varying filter. This filter matches the vocal tract transfer function, shaping the noise to produce specific spectral peaks (such as the high-frequency concentration characteristic of /s/).
   • A time-domain energy envelope is applied to modulate the noise amplitude, capturing the sharp attack, sustain, and decay of the consonant.

2. Voiced Fricatives (e.g., /z/, /v/, /ð/):

   • Voiced fricatives contain both vocal cord vibration and aerodynamic turbulence.
   • HNM handles this dual nature naturally via the intermediate placement of \(F_m\). The lower frequencies below \(F_m\) are synthesized using harmonics to capture the underlying low-pitch "buzz" (the voice bar), while the higher frequencies above \(F_m\) are synthesized using shaped noise to capture the continuous friction.

Synthesis in Parametric TTS

During parametric TTS synthesis, the engine's acoustic model (such as a neural network or hidden Markov model) outputs the acoustic parameters: \(F_0\), \(F_m\), harmonic spectral envelopes, and noise energy envelopes. The harmonic track is reconstructed using sinusoidal oscillators or Inverse Fast Fourier Transforms (IFFT), and the noise track is created through linear predictive filtering of white noise. Summing these two synchronized time-domain signals yields the final synthesized voiced or fricative speech.