Configure FFmpeg libx265 Adaptive Quantization

Adaptive Quantization (AQ) in the libx265 encoder is a critical feature that improves visual quality by distributing bits more effectively across different parts of a video frame, preventing blockiness in flat areas and gradient banding. This article provides a straightforward guide on how to configure the AQ mode and its strength in FFmpeg using the libx265 encoder parameters to optimize your HEVC/H.265 encodes.

Understanding the AQ Parameters

To configure Adaptive Quantization in libx265, you must pass specific parameters to the encoder using the -x265-params flag in your FFmpeg command. The two primary parameters are aq-mode and aq-strength.

1. AQ Mode (aq-mode)

The aq-mode parameter determines how the encoder distributes bits across different regions of a frame. You can choose from five different modes:

• aq-mode=0 (Disabled): Disables adaptive quantization entirely. This is rarely recommended as it often leads to visible artifacts in flat or dark areas.
• aq-mode=1 (Uniform AQ): Distributes bits across the entire frame. This is the default setting.
• aq-mode=2 (Auto-Variance AQ): Automatically decides the AQ strength per frame. It is highly effective at improving flat and dark areas.
• aq-mode=3 (Auto-Variance AQ with bias to dark scenes): Biases the distribution toward dark scenes to prevent color banding and blocking in shadows. This is highly recommended for high-quality movie encodes.
• aq-mode=4 (Edge-Directed AQ): Focuses on maintaining clean edges. This mode is particularly useful for animation, anime, and high-contrast content.

2. AQ Strength (aq-strength)

The aq-strength parameter adjusts the intensity of the chosen AQ mode. * Range: 0.0 to 3.0 * Default: 1.0 * A lower value (e.g., 0.8) reduces the effect of AQ, while a higher value (e.g., 1.2 to 1.5) increases the bit distribution variance, which can help eliminate banding in problematic videos but may increase the overall file size.

How to Configure AQ in the FFmpeg Command

To apply these settings, define them inside the -x265-params argument, separating multiple parameters with colons (:).

Basic Command Example

The following command encodes a video using libx265 with AQ Mode 3 (biased to dark scenes) and an AQ Strength of 1.2:

ffmpeg -i input.mp4 -c:v libx265 -crf 22 -x265-params aq-mode=3:aq-strength=1.2 -c:a copy output.mp4

Configuration Recommendations

Depending on your source material, use the following configurations for optimal results:

• For Movies and General Live-Action:

  -x265-params aq-mode=3:aq-strength=1.0

  Why: Prevents macroblocking and banding in dark scenes and shadows.

• For Anime and 2D Animation:

  -x265-params aq-mode=4:aq-strength=1.1

  Why: Preserves clean line art and sharp edges typical in animated content.

• For High-Bitrate Archiving:

  -x265-params aq-mode=2:aq-strength=1.0

  Why: Allows the encoder to dynamically adapt variance across frames without over-allocating bits to dark areas.