AVIF Performance on Dense Document Scans and Text

AVIF offers exceptional compression efficiency for complex photographic imagery, but applying it to dense document scans with fine typography reveals distinct trade-offs between file size and legibility. While the format can dramatically reduce file sizes compared to legacy formats like JPEG, its lossy compression architecture presents specific challenges for rendering micro-text, thin serifs, and high-frequency line art. Understanding how AVIF handles high-contrast, text-heavy scans requires evaluating its transform mechanics, filtering tools, color subsampling, and configuration requirements.

The Challenge of Fine Typography

Text documents consist of sharp, bi-level or high-contrast edges with high spatial frequency. Traditional lossy image encoders rely on discrete cosine transforms (DCT) or similar block transforms to discard high-frequency data that human vision typically ignores in natural scenes. In dense text documents, this high-frequency information constitutes the actual characters. Removing or approximating it leads directly to degraded legibility, smearing, and edge halos.

Smoothing and In-Loop Filtering Artifacts

The primary issue AVIF encounters with dense typography stems from the AV1 video codec's in-loop filtering toolset, specifically the Deblocking Filter and the Constrained Directional Enhancement Filter (CDEF).

In photographic content, these filters successfully eliminate block boundary artifacts and smooth out noise. On dense text, however, the encoder frequently interprets tiny serifs, punctuation marks, and diacritics as noise or block edges:

• Serif and Stroke Erosion: Thin horizontal crossbars (such as in the letters "e", "t", or "f") can disappear entirely at moderate compression ratios.
• Loss of Fine Punctuation: Commas, periods, and accents are frequently smoothed into the background or blurred into neighboring characters.
• Contrast Bleed: Instead of the high-frequency ringing common in JPEG, AVIF tends to produce watercolor-like smudges around characters, softening the transition between ink and paper.

The Impact of Chroma Subsampling

By default, many AVIF encoders convert input images to YUV 4:2:0. In document scans, this causes severe degradation:

• Colored Text and Highlights: Colored inks, annotations, or stamps lose edge definition and bleed into the surrounding page.
• Fringing Around Dark Text: Subsampling chroma channels introduces noticeable color artifacts around black text on white paper.

To preserve acceptable legibility on document scans, encoding in YUV 4:4:4 or native RGB is essential, even though this reduces overall compression gains.

Lossless AVIF vs. Dedicated Document Formats

AVIF supports a lossless mode, but it is not inherently optimized for the high-contrast, repetitive patterns found in scanned documents.

• Compared to PNG: Lossless AVIF often trails optimized PNG (using tools like Oxipng) in file size when dealing with clean, high-contrast black-and-white documents.
• Compared to JBIG2: For bi-level (monochrome) scans, JBIG2 vastly outperforms AVIF in both compression ratio and fidelity, as JBIG2 uses symbol dictionary matching designed specifically for recurring letterforms.
• Compared to JPEG XL: JPEG XL generally outperforms AVIF on document scans, offering superior edge preservation, better handling of high-frequency text boundaries, and stronger lossless compression ratios.

Optimal AVIF Configuration for Text Scans

When AVIF must be used for documents—such as in web delivery pipelines constrained to modern browser-supported formats—specific encoding parameters are necessary:

1. Pixel Format: Force yuv444 or RGB to prevent color bleeding on character edges.
2. Loop Filter Suppression: Lower or disable the deblocking filter strength to prevent the encoder from smoothing away fine strokes.
3. Low Quantization (CRF): Keep the Constant Rate Factor low (typically below 20–25) to avoid catastrophic text degradation.
4. Sharpness Tuning: Utilize encoder tunings optimized for sharpness or SSIM rather than perceptual metrics (like VMAF or tune=vmaf), which aggressively prioritize large-scale structures over micro-details.

Verdict

AVIF is not inherently suited for dense document scans featuring fine typography. While it can produce remarkably small files for mixed-content pages containing both photos and large text, pure or high-density document scans are prone to character dropouts and edge blurring. When fine text must be preserved accurately at high densities, formats such as JPEG XL, JBIG2 (for monochrome), or well-optimized PNG remain technically superior options.