AV1 Dynamic Resolution Scaling Without Keyframes

Yes, an AV1 encoder can downscale—and upscale—video resolution mid-stream without emitting a keyframe. This capability is made possible by a native AV1 feature known as Reference Frame Resampling (RFR). This overview explains how AV1 achieves inter-frame resolution changes, the technical mechanisms behind normative scaling, and the operational advantages this provides for real-time communications and adaptive bitrate streaming.

How AV1 Changes Resolution on Inter-Frames

In legacy video codecs like H.264 (AVC) and standard HEVC profiles, a change in frame dimensions typically invalidates previous reference buffers. To change resolution, the encoder must transmit a new Sequence Parameter Set (SPS) accompanied by an Instantaneous Decoder Refresh (IDR) or keyframe. Keyframes require significantly more bits than inter-frames (P-frames or B-frames), often creating sudden bandwidth spikes that can degrade network performance precisely when bandwidth is constrained.

AV1 eliminates this requirement through Reference Frame Resampling. Under the AV1 specification, an inter-frame can possess different pixel dimensions than the frames stored in its reference buffer. When the encoder decides to lower the resolution, it signals the new frame width and height directly in the frame header.

The Reference Frame Resampling (RFR) Mechanism

When decoding a frame that has a different resolution from its reference frames, the AV1 decoder applies a standardized scaling process:

1. Normative Rescaling: The AV1 specification defines a precise, normative 8-tap polyphase filtering algorithm for scaling reference frames. Because the scaling filter is strictly defined by the standard, both the encoder and decoder generate bit-identical interpolated reference pixels, preventing drift.
2. Motion Vector Scaling: Motion vectors pointing to a reference frame of a different resolution are automatically scaled by the decoder to align with the spatial dimensions of the target frame.
3. Buffer Management: Decoded frames of varying dimensions can be saved directly into the circular reference frame buffers (DPB). Subsequent frames can reference downscaled or full-scale frames without forcing a pipeline flush.

Practical Benefits for Real-Time Streaming

The ability to downscale mid-stream without keyframes provides critical advantages, particularly in real-time communication (RTC) frameworks like WebRTC:

• No Congestion Bitrate Spikes: When network throughput drops, an encoder usually needs to lower the resolution to match available bandwidth. Generating an IDR frame at this moment worsens network congestion. AV1 allows an immediate downscale using a lightweight delta frame.
• Lower Latency: Avoiding IDR frames eliminates buffer bloat and queue delays across network hops, keeping end-to-end latency minimal.
• Smooth Quality Degradation: Instead of dropping frames or stuttering while waiting for a keyframe negotiation, the video resolution smoothly shifts down and scales back up as network conditions dictate.

AV1 encoders like libaom, SVT-AV1, and specialized WebRTC implementations expose configuration flags to enable dynamic spatial resampling, allowing smooth, mid-stream resolution transitions entirely driven by inter-frames.