Automated Audio Cleaning for Text-to-Speech Corpora

High-quality Text-to-Speech (TTS) synthesis depends heavily on dry, pristine acoustic data free of room acoustics and physiological mouth noise. This article examines the automated pipelines used in modern machine learning workflows to systematically eliminate room reverberation, mouth clicks, and breath noises from large-scale voice datasets.

The Standard Audio Cleaning Pipeline Architecture

Building a production-ready TTS dataset from raw recordings involves an automated, sequential pipeline. Applying cleaning algorithms in the wrong order can cause artifacts; for instance, dereverberating an audio file after dynamic compression will amplify room reflections. The standard processing sequence typically follows this order:

1. Blind Dereverberation: Strips early reflections and late reverberation tails to create an anechoic baseline.
2. De-Clicking and Transient Repair: Identifies and interpolates over micro-transients caused by saliva and tongue movement.
3. Forced Alignment and Segmentation: Splits the audio into short utterances and aligns text tokens with phonemes.
4. Breath Detection and Attenuation: Attenuates or removes non-phonetic intake and exhale sounds during inter-word or sentence boundaries.
5. Loudness Normalization: Standardizes integrated loudness (e.g., -24 LKFS/LUFS) and true-peak levels across the corpus.

Automated Dereverberation Techniques

Room impulse response (RIR) introduces phase distortion and temporal smearing, which degrades the vocoder’s ability to generate crisp high-frequency details. Automated pipelines employ two primary classes of dereverberation:

• Weighted Prediction Error (WPE): Statistical approaches such as the nara_wpe library use iterative algorithms to invert the room transfer function without distorting the direct speech signal. WPE models late reverberation using long-term linear prediction, making it an industry standard for multi-channel and high-quality single-channel cleaning because it preserves speech naturalness and introduces minimal phase artifacts.
• Deep Neural Dereverberation: Modern pipelines frequently deploy neural models like VoiceFixer, DPCRN (Deep Complex Recurrent Network), or MetricGAN+. These networks operate in the complex short-time Fourier transform (STFT) domain to predict clean spectral masks or directly synthesize anechoic waveforms. While highly effective at stripping severe room reflections, they require careful tuning of regularization parameters to prevent "musical noise" and phoneme clipping.

Removing Mouth Clicks and Saliva Artifacts

Mouth clicks are short, high-frequency transients (typically 2 to 10 milliseconds in duration, concentrated above 4 kHz) caused by the separation of the tongue, lips, and palate. Automated removal relies on digital signal processing (DSP) rather than heavy neural networks to avoid altering underlying formant structures:

• Transient Detection: Algorithms analyze the derivative of the signal's energy envelope or employ wavelet decomposition to identify sudden, non-harmonic spikes that exceed local statistical variance.
• LPC-Based Interpolation: Once a click is localized, the pipeline removes the corrupted samples (typically 50–200 samples) and replaces them using Linear Predictive Coding (LPC) forward-backward interpolation or cubic spline reconstruction. This process repairs the waveform seamlessly without leaving an audible gap or phase discontinuity.

Breath Detection and Gating

While natural breathing can sometimes improve conversational TTS models, training standard acoustic models (such as FastSpeech 2 or VITS) requires precise control over non-speech sounds. Unintended loud inhalations create alignment errors and phoneme confusion.

• Forced-Alignment Masking: Tools like the Montreal Forced Aligner (MFA) align text transcripts with the acoustic signal. Intervals between words that do not map to distinct phonemes are flagged as silence or breath regions.
• VAD and Semantic Breath Classifiers: Modern pipelines run lightweight classification models, such as fine-tuned Silero VAD or CNN-based audio taggers, over unaligned regions to categorize pauses into true digital silence, room tone, or inhalations.
• Targeted Attenuation: Rather than hard-cutting breath segments—which causes unnatural gating and phase pops—automated scripts apply a dynamic downward expander or a cosine-ramped attenuation (lowering breath events by 12 dB to 18 dB) to preserve natural prosodic cadence while minimizing model bias.

Integrated End-to-End Solutions

Engineering teams often wrap these steps into continuous integration workflows using tools like Resemble Enhance, custom Python orchestrations (combining librosa, scipy, and torchaudio), or automated batch processing via headless sound engines. By combining statistical DSP for transient repair with deep learning models for acoustic dereverberation, these pipelines clean hundreds of hours of raw speech into studio-grade training corpora with minimal manual QA.