Frame-Parallel Decoding in AV1 Video Codec

Frame-parallel decoding is an architectural approach that enables a video decoder to process multiple frames simultaneously across different processor cores to maximize throughput. While traditional decoders parse and reconstruct video frames sequentially, modern high-resolution formats require parallelism to achieve real-time playback. The AOMedia Video 1 (AV1) bitstream syntax provides specific mechanisms to facilitate frame-level concurrency, while simultaneously imposing structural constraints governed by temporal reference dependencies and in-loop filtering.

What is Frame-Parallel Decoding?

Frame-parallel decoding allows a video player or hardware decoder to assign distinct, consecutive frames to separate worker threads. Unlike tile-based or slice-based parallelism—which splits a single frame into independent spatial regions—frame-parallel decoding distributes the processing of entire frames across time.

The primary challenge in frame-parallel processing is temporal dependency. Inter-predicted frames rely on pixel data from previously decoded reference frames. Consequently, a worker thread decoding Frame \(N+1\) cannot complete its reconstruction until the reference data from Frame \(N\) is available. Effective frame-parallel decoders manage this by overlapping operations: entropy decoding, motion vector parsing, and partial reconstruction occur concurrently across threads, resolving pixel dependencies dynamically as reference data finishes rendering.

AV1 Syntax Features Supporting Frame-Parallel Decoding

AV1 incorporates several syntax-level elements designed to make parallel processing efficient and computationally deterministic:

• CDF Updates Decoupling (disable_frame_end_update_cdf): In entropy coding, symbol probabilities are tracked using Cumulative Distribution Functions (CDFs). If each frame inherits the final adapted CDF state from the preceding frame, entropy decoding must remain strictly sequential. AV1 includes the disable_frame_end_update_cdf flag in the uncompressed frame header. When enabled, it prevents the frame from updating the base CDF state for subsequent frames, allowing multiple threads to entropy-decode frame headers and payloads in parallel without waiting for preceding frames to finish symbol parsing.
• Decoupled Decode and Display (show_existing_frame): AV1 separates frame decoding from frame presentation. A frame can be decoded into a reference buffer without being rendered immediately, or an existing buffer can be output without decoding new sample data using show_existing_frame. This explicit buffer-management syntax allows decoders to build lookahead graphs, decode out-of-order reference structures (such as hierarchical B-frames), and schedule worker threads efficiently.
• Explicit Reference Frame Refresh Signaling: The bitstream syntax explicitly declares which slots in the eight-frame reference buffer will be overwritten via the refresh_frame_flags. Because this is signaled at the very beginning of the frame header, worker threads know immediately which buffers will be occupied or released, avoiding race conditions among concurrent threads.
• Temporal Delimitation and Tile Groups: AV1 organizes bitstreams into Open Bitstream Units (OBUs). Temporal Delimiter OBUs distinctly mark time intervals, enabling a demuxer or parser thread to quickly slice the compressed bitstream into separate frame-level payloads and distribute them to idle worker threads before pixel decoding even begins.

Syntax Restrictions and Bottlenecks in AV1

While AV1 accommodates parallel designs, certain elements of its syntax restrict frame-level parallelism and introduce synchronization barriers:

• In-Loop Filtering Dependencies: AV1 employs three sequential in-loop filters: the Deblocking Filter, the Constrained Directional Enhancement Filter (CDEF), and the Loop Restoration filter. A reference frame is not legally valid for inter-prediction until all three filters have fully processed the boundary and inner pixels. Because Loop Restoration requires wider spatial pixel neighborhoods (up to \(64\times64\) or \(128\times128\) units), inter-dependent frames must wait for this intensive reconstruction step to finish before accessing reconstructed samples.
• Global and Warped Motion: AV1 supports complex inter-frame transformations, including affine and warped motion projection. These prediction modes depend on accurate reconstruction of previous frame boundaries and motion trajectories. This increases the complexity of reference frame availability tracking, forcing downstream threads to wait for exact sub-pixel boundaries to be fully calculated.
• CDF Synchronization Penalties: If an encoder does not set disable_frame_end_update_cdf, the decoder cannot parse the compressed data tokens of Frame \(N+1\) until Frame \(N\) has completely finished parsing all of its tiles and updated its CDF state. This re-serializes the front-end entropy parsing stage of the pipeline.
• Primary Reference Frame Selection: AV1 allows frames to select a primary_ref_frame from which to inherit initial probability states and motion vector references. If this reference is not yet available, the dependent frame's setup phase stalls, introducing thread latency.