AV1 Segment Maps and Localized Coding Features

Segment maps in the AV1 video codec are spatial control mechanisms that divide a video frame into distinct regions to apply tailored compression parameters. This article explains the technical architecture of AV1 segment maps, how they assign localized features—such as custom quantization, loop filtering, and reference frame selection—and the role they play in optimizing visual quality and compression efficiency through region-of-interest coding and perceptual rate control.

Understanding Segment Maps in AV1

In AV1, a segment map partitions a frame into up to eight distinct segments (labeled 0 through 7). Each coding block within the frame is assigned a segment ID. Rather than forcing global encoding parameters across an entire picture, the encoder uses these segment IDs to look up specific localized coding tools and parameters associated with each segment.

The assignment of blocks to segments is signaled at the frame level. To minimize bitrate overhead, AV1 employs two primary methods for updating the map:

• Temporal Updates: If a frame closely resembles the previous one, the decoder can inherit the previous frame's segment map. The bitstream only needs to transmit updates for regions where the segmentation changes.
• Spatial Signaling: When a fresh map is required (such as on keyframes or after scene cuts), the segment IDs are coded using tree-based entropy coding methods directly within the bitstream.

Localized Coding Features Controlled by Segments

Once a block is associated with a segment ID, several coding tools can be modified locally based on the segment's feature set. AV1 provides specific feature flags and data values for each segment:

1. Quantization Parameter (QP) Offsets

Segment maps allow the encoder to apply a localized delta QP (or alternate absolute QP) to specific regions. Blocks assigned to high-detail areas or regions of visual interest (such as human faces) can be assigned a lower QP to preserve fidelity. Conversely, complex textures with high spatial masking (like grass or water surfaces) can be given a higher QP to discard imperceptible details, saving valuable bits.

2. Loop Filtering Adjustments

AV1 includes multiple in-loop filters, including the deblocking filter and the Constrained Directional Enhancement Filter (CDEF). Through segment maps, the encoder can alter the loop filter strength per segment. Flat, low-texture areas prone to visible blocking artifacts can receive stronger filtering, while sharp edges and fine-line details can have loop filtering attenuated to avoid blurring.

3. Reference Frame Restrictions

Segments can restrict or dictate reference frame selection. By designating a segment to use only specific long-term or golden reference frames, the encoder can efficiently preserve static background elements across multiple frames without re-evaluating motion vectors across all potential reference buffers.

4. Transform and Skip Modes

Segment maps can enforce skip modes on a block-by-block basis. If a segment corresponds to completely static content (such as a static logo or an unchanging desktop background in screen content coding), the encoder can flag that segment to bypass residual transform coding entirely, transmitting only the indication that the block should be skipped.

Practical Applications in Video Encoding

The localized control provided by segment maps makes them essential for advanced AV1 encoding pipelines:

• Region of Interest (ROI) Encoding: In real-time video conferencing, face-detection models feed coordinate boundaries into the segment map, allowing the encoder to concentrate bit budget on faces while aggressively compressing the background.
• Perceptual Rate Control: Encoders utilizing perceptual metrics (like VMAF or SSIM) use segment maps to allocate bits based on Human Visual System (HVS) sensitivities rather than raw mathematical error.
• Screen Content Optimization: Computer screen captures contain both sharp text and complex UI elements. Segment maps isolate text boundaries from natural image overlays, preventing ringing artifacts around high-contrast characters while maintaining smooth gradients elsewhere.