Download and Train FFmpeg arnndn RNNoise Models

This article provides a concise guide on how to acquire and train neural network models for the arnndn (RNNoise) speech enhancement filter in FFmpeg. You will learn where to download pre-trained model files and the step-by-step process required to train your own custom recurrent neural network (RNN) models for tailored noise reduction.

Downloading Pre-Trained Models

The FFmpeg arnndn filter is based on the RNNoise project by Xiph.Org. If you do not want to train a model from scratch, you can download pre-compiled .rnnn model files that have been trained on diverse speech and noise datasets.

Where to Download

Pre-trained models are hosted on various open-source repositories. The most common standard models include: * cb.rnnn: Combined model (general-purpose noise reduction). * bd.rnnn: Broad noise reduction. * mp.rnnn: Music and voice preservation. * sh.rnnn: Statistical speech enhancement.

You can download these official files directly from the community-maintained RNNoise models GitHub repository or the original RNNoise repository.

How to Use a Downloaded Model in FFmpeg

Once you have downloaded a .rnnn file (for example, cb.rnnn), apply it to your audio file using the following FFmpeg command:

ffmpeg -i input.mp4 -af arnndn=model=/path/to/cb.rnnn output.mp4

Training a Custom Model

If the pre-trained models do not sufficiently reduce your specific background noise, you can train a custom model. Training requires a Linux-based environment, C compilation tools, Python, and Keras/TensorFlow.

Step 1: Clone the RNNoise Repository

Clone the official RNNoise source code repository and navigate into the directory:

git clone https://github.com/xiph/rnnoise.git
cd rnnoise

Step 2: Prepare Your Audio Training Data

You need two large sets of audio files in raw, 16-bit, 48 kHz, mono PCM format: 1. Clean Speech: Hours of clear voice recordings without any background noise. 2. Noise: Hours of pure background noise (e.g., office hum, fan noise, street traffic).

Step 3: Compile the Feature Extraction Tool

Before training, you must compile the C files to generate the feature extraction utility:

./autogen.sh
./configure
make src/denoise_training

This compilation creates the src/denoise_training executable, which extracts training features from your audio files.

Step 4: Extract Features

Use the compiled binary to mix your clean speech and noise files, then extract the training features. Execute the binary by passing your clean speech and noise PCM files as arguments:

./src/denoise_training clean_speech.pcm noise.pcm 50000000 training_data.f32

Note: 50000000 is the number of training sequences to generate. training_data.f32 is the output file containing the extracted features.

Step 5: Train the Model Using Python

Navigate to the training directory and run the Keras training script. Ensure you have TensorFlow and Keras installed in your Python environment.

cd training
python rnn_train.py ../training_data.f32

This script processes the extracted features and trains the recurrent neural network. It will output weight files (typically in .h5 or .py structure format).

Step 6: Export the Model to RNNoise Format

After training is complete, compile the final weights back into the C source code structure to output the compatible .rnnn file.

Use the provided utility script in the training directory to convert the Keras weights into C header files:
```
python dump_rnn.py weights.h5 ../src/rnn_data.c ../src/rnn_data.h
```
Recompile the RNNoise library:
```
cd ..
make clean
make
```
Use the compiled library or extraction scripts to serialize these weights into a single .rnnn file. You can now pass this newly created custom .rnnn file directly to FFmpeg’s arnndn filter.