AV1 IntraBC: Locating Repetitive UI Patterns

The AV1 video codec incorporates a specialized screen content coding feature known as Intra Block Copy (IntraBC) to dramatically improve compression efficiency for synthetic graphics, remote desktop feeds, and application interfaces. This article explains what the IntraBC tool is, its underlying operating principles, and how the search and block vector mechanisms detect and compress identical, repetitive user interface (UI) elements within a single frame.

What Is the IntraBC Tool in AV1?

Traditional intra-prediction predicts pixel values in a current block by extrapolating from the boundary pixels of immediately adjacent, already reconstructed blocks. While effective for natural video with smooth gradients and camera noise, this spatial extrapolation fails to compress screen content efficiently. Modern computer displays and software UIs consist of crisp text, geometric borders, solid colors, and identical icons placed at distant coordinates across the display.

IntraBC bridges this gap by applying the principles of inter-frame motion compensation inside a single frame. Instead of referencing an entirely different frame in time, IntraBC treats the current, partially decoded frame as a reference picture. It allows the encoder to copy matching blocks of pixels from previously decoded regions directly into the current block, bypassing the limitations of edge-based spatial prediction.

How IntraBC Locates Repetitive UI Patterns

To locate and reuse repeating UI components—such as recurring glyphs in text, toolbar buttons, list indicators, or window edges—the IntraBC process relies on a structured sequence of vector search, matching criteria, and vector prediction.

1. Integer-Pixel Block Vector Search

When evaluating a coding block, the AV1 encoder performs a search across the reconstructed areas of the current frame to identify identical or near-identical pixel arrangements.

• Block Vectors (BVs): The spatial displacement between the current block and the reference block is represented by a Block Vector rather than a Motion Vector.
• Full-Pel Resolution: Standard video motion vectors use sub-pixel (half-pel or quarter-pel) precision and interpolation filters to estimate smooth movement. IntraBC restricts block vectors to integer-pixel (full-pel) precision. This design prevents the blurring of sharp graphic edges and minimizes computational overhead when copying UI assets.

2. Distortion Metrics and Pattern Matching

During the search stage, candidate blocks within the search window are evaluated using rate-distortion optimization (RDO) alongside distortion metrics such as:

• Sum of Absolute Differences (SAD) or Sum of Squared Errors (SSE) to verify exact or near-exact pixel equality.
• Cost Evaluation: The encoder balances the fidelity of the match against the bit cost required to signal the displacement vector. Because computer UI elements are digital duplicates, the residual error (the difference between the reference and the current block) is frequently zero, allowing the encoder to skip residual coding entirely.

3. Search Range Constraints

The search for matching UI patterns must strictly follow the decoding order:

• The encoder can only search regions of the frame that have already been reconstructed.
• The search is bounded by hardware constraints to keep the decoding pipeline feasible. AV1 restricts the valid reference memory to an area that avoids dependencies on unfiltered loop filter pixels and respects tile boundaries, ensuring memory buffers remain within specified limits.

4. Vector Prediction and Regular Spacing

UI elements often appear in grid-like, evenly spaced layouts (such as rows of menu items or lines of text). AV1 leverages this spatial regularity through block vector prediction:

• Vector Candidates: The encoder derives predicted block vectors from spatial neighbors that also utilized IntraBC.
• Residual Encoding: When an entire row of identical UI icons repeats with fixed intervals, the difference between their block vectors is minimal or zero. AV1 encodes only the block vector difference (BVD), drastically reducing the bitstream overhead needed to replicate patterns across the screen.