AV1 LTR Frame Acknowledgment Signaling

In real-time AV1 video streaming, receivers use transport-level feedback to confirm the receipt of Long-Term Reference (LTR) frames to maintain stream resilience without relying on bandwidth-heavy keyframes. This article explains how an AV1 receiver identifies an LTR frame, generates the appropriate RTP Control Protocol (RTCP) feedback messages, and communicates frame integrity back to the encoder to establish a shared reference state.

Identification via RTP Header Extensions

Before an acknowledgment can occur, the receiver must recognize that a decoded frame is intended as an LTR. In AV1 video transmission over RTP, this is handled through RTP header extensions:

• AV1 Dependency Descriptor: The sender attaches a Dependency Descriptor to the RTP packet containing the AV1 payload. This descriptor indicates the frame's structural role, its unique picture ID (picture_id), and instructions on whether the frame updates a specific reference buffer slot.
• Buffer Slot Allocation: AV1 supports up to eight reference frame slots in its decoder buffer. The descriptor signals which of these slots should store the current frame as a long-term reference.

RTCP Feedback Signaling

Once the receiver successfully decodes the designated frame and commits it to the designated reference buffer slot, it communicates this success back to the sender using RTCP:

• Payload-Specific Feedback (PSFB): Receivers use RTCP PSFB messages (defined in RFC 4585) to send feedback targeting specific video frames.
• Reference Picture Selection Indication (RPSI): The receiver generates an RPSI feedback message containing the native payload identifier or picture ID of the decoded LTR frame. This acts as an explicit positive acknowledgment (ACK).
• Layer Refresh Request (LRR): When combined with scalable video coding (SVC) modes commonly used in AV1, receivers can issue an LRR message to signal the sender to recover state or to confirm that a base-layer reference is available.

Negative Acknowledgments and Loss Recovery

If an LTR frame is corrupted or dropped in transit:

• Implicit NACK: The receiver does not send an acknowledgment for the missing frame. The sender maintains its previous safe reference state until an explicit ACK arrives.
• Picture Loss Indication (PLI): If the decoder experiences packet loss and cannot recover using existing reference frames, it sends a PLI or Full Intra Request (FIR) to prompt the encoder for a recovery frame or an updated LTR refresh.

Encoder-Side Synchronization

When the sender receives the RTCP acknowledgment containing the valid picture ID:

1. Buffer Verification: The encoder marks the corresponding reference buffer as validated and synchronized with the remote receiver.
2. Predictive Coding: In subsequent frames, the encoder can reference this confirmed LTR frame, ensuring that packet loss occurring on intermediate short-term reference frames does not cause decoding artifacts or freeze playback.