How HiFi-GAN Generates Artifact-Free TTS Audio

HiFi-GAN (High-Fidelity Generative Adversarial Network) achieves state-of-the-art text-to-speech synthesis by utilizing a generator coupled with two complementary discriminator architectures: the Multi-Period Discriminator (MPD) and the Multi-Scale Discriminator (MSD). While traditional neural vocoders often produce metallic buzzes, phase issues, or muffled sound, HiFi-GAN eliminates these artifacts by evaluating raw audio waveforms across both disjoint periodic intervals and varying temporal resolutions. This dual-discriminator setup forces the generator to capture both fine-grained harmonic structures and global acoustic continuity efficiently.

The Role of the Multi-Period Discriminator (MPD)

Natural speech is predominantly periodic, composed of fundamental frequencies, pitch contours, and harmonic structures. Standard 1D convolutional discriminators often fail to track these repetitive cycles because consecutive samples in raw audio are separated by varying periodic intervals depending on pitch.

The MPD addresses this limitation by reshaping 1D raw audio into 2D representations based on specific prime periods (such as 2, 3, 5, 7, and 11). By applying 2D convolutions over these structured grids, each sub-discriminator within the MPD independently examines periodic patterns across channels while inspecting adjacent samples within each cycle.

Because prime periods avoid overlapping receptive fields, the MPD captures isolated harmonic components without interference. This mechanism directly prevents periodic distortions—such as unnatural pitch fluctuations, robotic timbres, and metallic buzzing—which are common failure modes in conventional GAN-based vocoders.

The Role of the Multi-Scale Discriminator (MSD)

While the MPD specializes in periodic and harmonic features, speech also contains non-periodic elements, transient bursts (such as consonant plosives), and broader low-frequency structures. The Multi-Scale Discriminator handles these aspects by evaluating the audio waveform across multiple temporal scales.

The MSD consists of three sub-discriminators operating on downsampled versions of the audio signal:

1. The original raw audio waveform.
2. The waveform downsampled by a factor of two using average pooling.
3. The waveform downsampled by a factor of four using average pooling.

By assessing the audio at these varying resolutions, the MSD evaluates long-term temporal dependencies and low-frequency components without requiring an excessively large receptive field. This architecture prevents broad phase misalignment, hollow-sounding reverberations, and background hiss.

Dual-Architecture Synergy and Feature Matching Loss

The combination of MPD and MSD provides full coverage of the audio spectrum. The MPD regulates high-frequency fidelity and periodic pitch alignment, while the MSD preserves global structural coherence and smooth transitions.

To stabilize training and prevent mode collapse, HiFi-GAN incorporates a feature matching loss alongside the adversarial loss. Instead of solely penalizing the generator based on whether the discriminator classifies audio as real or fake, the feature matching loss minimizes the difference between feature maps extracted from intermediate layers of both discriminators for real and synthesized speech.

By constraining intermediate representation patterns across multiple periods and scales, the generator receives rich, localized gradient feedback. This structural collaboration enables HiFi-GAN to synthesize high-sample-rate, artifact-free speech with exceptional computational speed.