Configure libx264 Adaptive Quantization in FFmpeg

Adaptive quantization (AQ) is a critical encoding technique in the libx264 encoder that improves visual quality by distributing bits more intelligently across different areas of a video frame. This article provides a straightforward guide on how to configure the adaptive quantization mode (-aq-mode) and strength (-aq-strength) in FFmpeg to optimize your video compression.

Understanding the Adaptive Quantization Parameters

To control adaptive quantization in libx264, FFmpeg utilizes two main parameters: -aq-mode and -aq-strength.

1. AQ Mode (-aq-mode)

This parameter defines the algorithm used to distribute bits. You can choose from four available modes:

  • 0 (Disabled): Turns off adaptive quantization entirely. This is rarely recommended as it can lead to severe blocking artifacts in flat areas.
  • 1 (Variance AQ - Default): Distributes bits based on local complexity. It allocates more bits to complex, detailed areas and fewer bits to flat areas.
  • 2 (Auto-Variance AQ): Attempts to prevent banding and blocking in flat areas (like skies or walls) by shifting bits from complex areas to flat areas.
  • 3 (Auto-Variance AQ with Dark Bias): An extension of mode 2 that biases bit distribution toward dark scenes. It prevents color banding, blocking, and washing out in shadows and low-light footage.

2. AQ Strength (-aq-strength)

This parameter dictates how aggressively the selected AQ mode redistributes bits.

  • Default value: 1.0
  • Range: Typically between 0.0 and 1.5 (though higher values are technically possible).
  • A lower strength (e.g., 0.5) reduces the variance in bit distribution, while a higher strength (e.g., 1.3) forces a stronger redistribution of bits to the targeted areas.

How to Apply AQ Settings in FFmpeg Commands

To configure these settings, pass the -aq-mode and -aq-strength flags within your FFmpeg command line.

Basic Syntax

ffmpeg -i input.mp4 -c:v libx264 -aq-mode 2 -aq-strength 1.2 output.mp4

1. Best for General Content and Movies (Default)

If you are encoding standard live-action video with balanced lighting, the default settings usually work best.

ffmpeg -i input.mp4 -c:v libx264 -aq-mode 1 -aq-strength 1.0 output.mp4

2. Best for Dark Scenes and Horror Movies

If your video has many dark scenes, night footage, or shadows where blockiness is highly visible, use mode 3.

ffmpeg -i input.mp4 -c:v libx264 -aq-mode 3 -aq-strength 1.1 output.mp4

3. Best for 2D Animation and Anime

Flat colors and gradients in animation are highly susceptible to color banding. Mode 2 helps smooth these gradients out.

ffmpeg -i input.mp4 -c:v libx264 -aq-mode 2 -aq-strength 0.9 output.mp4