Causes of Metallic Artifacts in GAN Vocoders

Generative Adversarial Network (GAN) vocoders, such as HiFi-GAN and MelGAN, generate high-fidelity speech rapidly by converting mel-spectrograms into time-domain waveforms. However, despite their speed and perceptual quality, these models frequently introduce unnatural metallic timbres, robotic buzzing, or high-frequency ringing into the synthesized audio. This article examines the primary architectural and mathematical causes of these artifacts, detailing how upsampling issues, phase generation errors, harmonic mismatches, and discriminator limitations degrade synthetic speech.

Checkerboard Artifacts from Transposed Convolutions

GAN-based vocoders use transposed 1D convolutions to progressively upsample low-resolution mel-spectrogram frames into high-frequency raw audio samples. When the kernel size is not perfectly divisible by the stride, or when the convolutional weights do not overlap evenly, periodic patterns emerge in the output waveform. In audio, these periodic spikes—often called "checkerboard artifacts"—manifest directly as a persistent, high-frequency metallic hum or buzz.

Sub-Nyquist Aliasing

The non-linear activation functions (such as LeakyReLU or Snake) applied after upsampling operations frequently introduce high-frequency components that exceed the Nyquist frequency of intermediate feature maps. When these frequencies fold back into the audible spectrum without proper anti-aliasing filtering, they produce intermodulation distortion. This distortion alters the upper harmonics of speech, giving fricatives, vowels, and voiced consonants an unnatural, tinny sheen.

Phase Reconstruction and Temporal Discontinuities

Mel-spectrograms discard the phase information of the original audio signal, requiring the generator to reconstruct it from magnitude data alone. Because non-autoregressive GAN vocoders predict the entire waveform in parallel rather than sample-by-sample, small errors in local sample alignment lead to phase cancellation or sudden phase jumps between adjacent frames. When the ear detects rapid, inconsistent phase changes across frames, it perceives the result as a mechanical, raspy, or buzzing sensation rather than a smooth human voice.

Harmonic Misalignment and F0 Inconsistencies

Voiced speech consists of an excitation signal generated by the vocal cords, represented by a fundamental frequency (F0) and its integer harmonics. If the generator fails to precisely track the pitch contours embedded in the mel-spectrogram, the synthesized harmonics drift slightly out of tune with one another. This acoustic phenomenon mimics a comb filter or excessive chorus effect, smearing harmonic peaks and introducing an unmistakable metallic resonance to sustained vowel sounds.

Discriminator Resolution Limits

Most modern GAN vocoders rely on Multi-Scale Discriminators (MSD) and Multi-Period Discriminators (MPD) to evaluate audio across various frequency bands and periodic intervals. However, these discriminators operate on downsampled representations or specific sub-sampled periods, which can leave spectral "blind spots." If a discriminator cannot adequately evaluate the fine structures within narrow high-frequency bands, the generator will produce spurious, unpenalized acoustic noise in those regions, resulting in background buzzing.