Libaom vs HEVC: Which is better at low bitrates?

Evaluating video codecs at low bitrates is crucial for optimizing streaming performance under constrained bandwidth conditions. This article provides a comparative analysis of the visual quality delivered by the Reference Encoder for AV1 (libaom) against High Efficiency Video Coding (HEVC/H.265) when operating at low bitrates. We will examine objective metrics, subjective visual artifacts, and encoding efficiency to determine which codec holds the advantage when data budgets are tight.

Objective Quality Metrics: AV1 vs HEVC

When comparing libaom and HEVC at lower bitrates (typically defined as bitrates where traditional codecs start showing noticeable compression artifacts), objective video quality metrics like PSNR, SSIM, and Netflix’s VMAF show a distinct trend.

• VMAF Scores: Broadly, AV1 via libaom achieves higher VMAF scores than standard HEVC encoders (like x265) at equivalent low bitrates. This gap is often quantified as a 15% to 25% bitrate saving for the same perceptual quality.
• BD-Rate Savings: Bjontegaard-Delta (BD) rate measurements consistently demonstrate that libaom can maintain acceptable structural integrity and color accuracy at bitrates where HEVC begins to degrade rapidly.

Subjective Visual Performance and Artifacts

While numbers provide a baseline, the human eye perceives compression artifacts differently depending on the architectural design of the codec. At low bitrates, the visual characteristics of libaom and HEVC diverge significantly.

Libaom (AV1) Visual Characteristics

libaom utilizes advanced coding tools like larger block sizes (up to 128x128), sophisticated intra-prediction modes, and a specialized restoration filter.

• Smoothness over Blockiness: At low bitrates, libaom tends to favor smoothing out fine textures over introducing harsh blocking artifacts. This results in a cleaner, albeit softer, image.
• Film Grain Synthesis: One of AV1’s major advantages at low bitrates is its ability to strip out synthetic or organic film grain during encoding and recreate it at the decoder side. This prevents the encoder from wasting precious bits trying to compress random noise.

HEVC Visual Characteristics

HEVC relies on a Coding Tree Unit (CTU) structure up to 64x64 and an in-loop deblocking filter paired with Sample Adaptive Offset (SAO).

• Blocking and Blurring: When pushed to very low bitrates, HEVC can struggle to maintain uniform texturing, leading to visible blocking around complex motion areas and a distinct “smudging” effect caused by the SAO filter trying to smooth over errors.
• Detail Retention vs Distortion: HEVC may try to preserve edge sharpess slightly harder than AV1, but this often introduces ringing artifacts or pixelation around high-contrast edges when the bitrate budget is exhausted.

Computational Trade-offs

While libaom generally delivers superior visual quality and fewer objectionable artifacts than HEVC at low bitrates, it comes with a significant caveat: encoding complexity.

libaom requires substantially more computational resources and time to encode video compared to mature HEVC encoders like x265. Even though hardware acceleration for AV1 decoding is now widespread, encoding at high efficiency levels with libaom demands heavy CPU utilization. Therefore, the visual quality advantage of AV1 at low bitrates must be weighed against the increased energy and time costs required during the compression phase.