AV1 Scalable Video Coding: Inter-Layer Prediction

AV1 natively supports Scalable Video Coding (SVC) without requiring a separate extension profile or specialized inter-layer prediction syntax. Instead of introducing isolated prediction modes, AV1 achieves inter-layer prediction by combining its flexible reference frame architecture with normative reference picture resampling. This approach allows spatial, temporal, and quality enhancement layers to efficiently predict data from lower layers using standard inter-frame coding tools.

Reference Frame Management and Buffer Allocation

In AV1, the decoder maintains an eight-slot reference frame buffer pool. When operating in scalable modes, the encoder explicitly designates which decoded frames from base or lower layers are retained in these buffer slots.

When an enhancement layer frame is decoded, the bitstream configures its inter-frame references—such as LAST_FRAME, GOLDEN_FRAME, or ALTREF_FRAME—to point directly to lower-layer frames stored in the pool. This allows inter-layer prediction to function identically to standard temporal inter-frame prediction from the perspective of the motion compensation pipeline.

Reference Picture Resampling for Spatial Scalability

The primary enabler of spatial inter-layer prediction in AV1 is its native Reference Picture Resampling (normative downscaling and upscaling) feature. In earlier standards, such as H.264/SVC or HEVC/SHVC, inter-layer spatial prediction required dedicated texture-upsampling filters and distinct prediction syntax.

AV1 eliminates this complexity:

• On-the-Fly Scaling: When a higher-resolution spatial enhancement layer selects a lower-resolution base-layer frame as a reference, the decoder automatically applies AV1's normative phase-adaptive 8-tap upscaling filter.
• Unified Motion Compensation: Once the lower-layer reference buffer is rescaled to match the current frame's resolution, the enhancement layer performs motion search, block partitioning, and residual coding using ordinary inter-prediction mechanisms.
• Zero-Motion Vectors: When the spatial enhancement layer directly aligns with the base layer, motion vectors are commonly coded as zero, utilizing the rescaled lower-layer content directly as the predictor.

Quality (SNR) Scalability

For SNR or fidelity scalability, the spatial resolution remains identical between the base layer and enhancement layers. Inter-layer prediction in this scenario requires no scaling operations:

• The base layer frame is decoded and stored in a reference buffer slot.
• The enhancement layer points to this buffer slot at the exact same spatial dimensions.
• Prediction occurs directly via standard inter-coding, allowing the enhancement layer to transmit only refined residual data to improve visual quality.

Scalability Structure Signaling

AV1 communicates dependency rules and prediction paths through the scalability_structure syntax, embedded within the Sequence Header Open Bitstream Unit (OBU) or metadata OBUs. This structure defines:

• The total number of spatial and temporal layers.
• The reference picture marking patterns.
• Explicit frame-dependency trees, ensuring decoders know which lower-layer buffers must be retained for cross-layer prediction without buffer corruption.

By treating lower-layer frames simply as alternative references within a unified frame-management system, AV1 streamlines decoder complexity while enabling robust cross-layer prediction for spatial, temporal, and SNR scalability.