Filtering Inconsistent Vocal Fry in TTS Datasets

Building a high-quality, multi-day Text-to-Speech (TTS) dataset requires strict acoustic consistency, yet voice talent naturally experiences physiological changes that introduce unintended vocal fry—also known as creaky voice—across different recording sessions. This article details the computational and procedural workflows dataset curators use to detect, measure, and filter out inconsistent vocal fry. By combining acoustic feature extraction, machine learning classification, session baseline tracking, and targeted pruning strategies, audio engineers ensure the synthetic voice maintains stable timbre and pitch across thousands of generated utterances.

Acoustic Indicators of Vocal Fry

Vocal fry occurs when the vocal folds close tightly and vibrate irregularly at a very low frequency, creating a distinct rattling or popping sound. To identify this algorithmically, curators extract several key acoustic features from the raw audio:

• Sub-Harmonic F0 and Irregular Pitch Periods: Fry typically displays an extremely low fundamental frequency (often dropping below 70 Hz for men and 80 Hz for women) along with high period-to-period variability (jitter).
• Harmonics-to-Noise Ratio (HNR): Creaky voice introduces aperiodic noise into the signal, causing a noticeable drop in the HNR, particularly in the lower frequency bands.
• Amplitude Perturbation (Shimmer): Rapid, irregular variations in peak amplitude from one glottal cycle to the next serve as a reliable marker of glottal instability.
• Harmonic Amplitudes (H1 - H2):** The difference between the amplitude of the first harmonic and the second harmonic reflects glottal opening behavior. A depressed or negative H1*-H2* value indicates tightly closed vocal folds characteristic of vocal fry.

Automated Detection Pipelines

Curators rarely inspect tens of hours of speech manually. Instead, they deploy automated pipelines combining signal processing tools with machine learning models:

1. Feature Extraction Engines: Open-source speech processing tools (such as Praat/Parselmouth and openSMILE) calculate frame-by-frame metrics including Cepstral Peak Prominence (CPP), local jitter, and subharmonic ratios.
2. Frame-Level Classifiers: Lightweight binary classifiers (such as Random Forests or Support Vector Machines trained on acoustic features) or deep learning architectures (like 1D-CNNs or fine-tuned wav2vec 2.0 checkpoints) classify speech frames as modal or creaky.
3. Boundary Aggregation: Because vocal fry frequently manifests at the ends of phrases (terminal fry) due to falling subglottal pressure, detection scripts specifically calculate the percentage of fry present in the final 200–500 milliseconds of each phrase.

Session Baseline Normalization and Drift Tracking

Vocal fry fluctuates based on hydration, fatigue, time of day, and recording environment. Curators mitigate multi-day inconsistency through baseline calibration:

• Standardized Calibration Phrases: Voice actors record an identical set of warm-up sentences at the beginning, middle, and end of every recording day. Curators analyze these anchor recordings to establish that day's baseline vocal tract configuration.
• Session Outlier Detection: Utterances are grouped by recording date and time. Curators generate statistical distributions of fry percentage across all files in a batch. If Day 3 exhibits a 15% increase in terminal fry compared to Day 1, that entire session is flagged for systematic shift rather than treated as random noise.

Remediation and Filtering Workflows

Once inconsistent vocal fry is pinpointed, curators take distinct actions depending on the severity and location of the creak:

• Automated Dataset Pruning: Utterances exceeding an acceptable fry threshold—especially mid-phrase fry that directly harms neural vocoder training—are systematically discarded from the corpus if total phoneme coverage remains adequate.
• Automated Trimming: When fry is strictly isolated to the final trailing syllables or unvoiced release of an utterance, curators can programmatically trim the trailing frames, provided it does not alter word intelligibility or natural prosodic decay.
• Targeted Re-Recording Queues: For scripts containing rare diphones or vital phonetic transitions, rejected takes are automatically compiled into a dynamic pick-up list. The voice actor re-records these specific prompts in a subsequent session under monitored conditions to replace the degraded audio.