How Opus Forward Error Correction Works

The Opus audio codec utilizes Forward Error Correction (FEC) to maintain high-quality audio transmission over unstable networks with high packet loss, such as VoIP and real-time streaming. This article explains how Opus implements in-band FEC, how the decoder reconstructs missing packets using redundant data, and how the system dynamically adapts to changing network conditions to prevent audio glitches and dropouts.

The Concept of In-Band FEC in Opus

Unlike traditional out-of-band error correction methods that send separate redundant packets, Opus uses “in-band” FEC. This means the redundant data is embedded directly within the primary audio packets.

When the encoder decides to use FEC, each packet (let’s call it Packet N) carries two pieces of information: 1. The primary, high-quality audio payload for the current time frame (N). 2. A secondary, highly compressed, lower-bitrate representation of the audio from the previous time frame (N-1).

By packing the redundant data of the previous frame into the current frame, Opus ensures that the receiver can still recover the lost audio of frame N-1 if it is dropped during transmission, provided frame N arrives successfully.

The Recovery Process at the Decoder

When audio packets are transmitted over a network, they are received by the Opus decoder. The recovery process handles two main scenarios:

• Scenario A: No Packet Loss. If Packet N-1 and Packet N both arrive safely, the decoder plays the high-quality primary payload of Packet N-1, followed by the high-quality primary payload of Packet N. The redundant FEC data inside Packet N is simply discarded.
• Scenario B: Packet N-1 is Lost. If Packet N-1 fails to arrive but Packet N arrives safely, the decoder detects the gap. Instead of playing silence or guessing the missing audio, the decoder extracts the low-bitrate FEC data of N-1 from Packet N. It decodes this redundant data to fill the gap, resulting in a seamless transition with only a minor, often imperceptible, drop in audio quality.

Dynamic Adaptation and Bandwidth Management

Opus does not use FEC constantly, as sending redundant data requires additional bandwidth. Instead, it dynamically manages FEC based on real-time network feedback (usually provided via RTCP packets in VoIP applications):

1. Packet Loss Threshold: The encoder monitors the network’s packet loss rate. FEC is typically only activated when packet loss exceeds a specific threshold (e.g., 2% to 5%).
2. Bitrate Availability: Because FEC increases the size of the packets, the encoder will only enable it if there is sufficient configured bitrate to support both the primary payload and the redundant payload without causing further network congestion.
3. Speech vs. Music: Opus prioritizes FEC for speech (using its SILK mode) where intelligibility is critical, adjusting the level of redundancy based on the severity of the connection issues.

Opus FEC vs. Packet Loss Concealment (PLC)

It is important to distinguish FEC from Packet Loss Concealment (PLC), though Opus utilizes both:

If both Packet N-1 and Packet N are lost, Opus cannot use FEC to recover the audio. In this scenario, the decoder falls back to PLC to smoothly fade out the audio and minimize the impact of the consecutive losses on the listener.