What Is FFmpeg -aq-mode and How to Configure It

This article explains the purpose of the -aq-mode (Adaptive Quantization Mode) option in FFmpeg’s libx264 and libx265 encoders. It covers how adaptive quantization works to distribute bits across different parts of a video frame, details the various mode settings available, and provides practical configuration examples to help you optimize your video encoding quality.

What is Adaptive Quantization (-aq-mode)?

Adaptive Quantization (AQ) is a feature in video encoders like x264 and x265 that improves visual quality by distributing bits more smartly within a video frame.

Human eyes are sensitive to artifacts (like blocking or banding) in flat, smooth, or dark areas (such as skies, walls, or shadows). However, our eyes do not easily notice loss of detail in highly textured, fast-moving, or complex areas (such as grass, water ripples, or confetti).

Without AQ, encoders tend to spend too many bits on complex areas and too few bits on flat areas, leading to visible banding or pixelation in smooth gradients. The -aq-mode setting allows the encoder to shift bits away from complex textures and allocate them to flat and dark areas, resulting in a cleaner, more visually appealing output.

The Available AQ Modes

When configuring -aq-mode in FFmpeg, you can choose from several integer values, each representing a different strategy for bit distribution:

• 0 (Disabled): Turns off adaptive quantization completely. Bits are distributed uniformly. This is generally not recommended as it often leads to severe banding in gradients.
• 1 (Variance AQ): The default mode for libx264. It uses local variance to distribute bits. It identifies complex versus flat regions and shifts bits accordingly.
• 2 (Auto-Variance AQ): The default mode for libx265. It calculates the variance of the entire frame before redistributing bits. This mode is highly effective at cleaning up flat surfaces and is often preferred for clean content like animation or anime.
• 3 (Auto-Variance AQ with Bias to Dark Scenes): An advanced mode that works similarly to mode 2 but adds an extra bias toward dark areas. Dark scenes in video often suffer from “color banding” or “blocking.” Mode 3 forces the encoder to allocate extra bits to shadow details to keep dark scenes clean.

How to Configure -aq-mode in FFmpeg

To use -aq-mode in FFmpeg, you must specify the flag along with its corresponding value during encoding. This option is passed to the x264 or x265 encoder.

Example 1: Standard x264 Encoding (Mode 1)

For general live-action video using the H.264 codec, Mode 1 is highly reliable:

ffmpeg -i input.mp4 -c:v libx264 -aq-mode 1 -crf 22 output.mp4

Example 2: Optimizing Anime or Animation with x264 (Mode 2)

For cartoons, anime, or flat graphic screencasts where flat colors are prominent:

ffmpeg -i input.mp4 -c:v libx264 -aq-mode 2 -crf 20 output.mp4

Example 3: Enhancing Dark Scenes with x265 (Mode 3)

If you are encoding a movie with many dark scenes or night footage using the HEVC/H.265 codec:

ffmpeg -i input.mp4 -c:v libx265 -aq-mode 3 -crf 24 output.mp4

Adjusting AQ Strength (-aq-strength)

Alongside -aq-mode, you can configure how aggressively the bit redistribution is applied using the -aq-strength flag.

• The default strength is 1.0.
• Acceptable values range from 0.0 to 3.0.
• If you notice that flat areas still have too many artifacts, you can increase the strength (e.g., -aq-strength 1.3). If flat areas look great but high-detail areas are becoming too blurry, you can lower the strength (e.g., -aq-strength 0.8).

ffmpeg -i input.mp4 -c:v libx264 -aq-mode 3 -aq-strength 1.2 -crf 20 output.mp4