Configure hevc_vaapi Rate Control and Quality in FFmpeg

This article provides a straightforward guide on how to configure rate control modes and quality options when using the hardware-accelerated hevc_vaapi encoder in FFmpeg. You will learn how to set Constant QP (CQP), Constant Bitrate (CBR), Variable Bitrate (VBR), and Constant Quality (ICQ) modes, along with adjusting speed and compression presets.

Basic VAAPI Pipeline Structure

Before configuring rate control, ensure your FFmpeg command is structured to use VAAPI hardware acceleration. A typical transcode command looks like this:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.mp4 -vf "scale_vaapi=format=nv12" -c:v hevc_vaapi [quality/rate control options] output.mp4

Configuring Rate Control Modes

The hevc_vaapi encoder uses the -rc_mode flag to define how bitrate and quality are managed.

1. Constant QP (CQP)

Constant QP mode applies a fixed quantization parameter to every frame. This mode prioritizes consistent image quality over file size.

• Flag: -rc_mode CQP
• Quality Setting: Use -qp (ranges from 0 to 51, where lower values mean higher quality and larger file sizes; 20-25 is standard).

Example:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i input.mp4 -c:v hevc_vaapi -rc_mode CQP -qp 23 output.mp4

2. Constant Bitrate (CBR)

CBR targets a strict, constant output bitrate. It is ideal for streaming scenarios where bandwidth is limited and predictable.

• Flag: -rc_mode CBR
• Bitrate Setting: Use -b:v to set the target bitrate and match it with -maxrate.

Example:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i input.mp4 -c:v hevc_vaapi -rc_mode CBR -b:v 5M -maxrate 5M output.mp4

3. Variable Bitrate (VBR)

VBR allows the bitrate to fluctuate depending on the complexity of the video scene, maintaining a target average while capping the maximum spike.

• Flag: -rc_mode VBR
• Bitrate Setting: Use -b:v for the average target bitrate and -maxrate for the maximum peak.

Example:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i input.mp4 -c:v hevc_vaapi -rc_mode VBR -b:v 5M -maxrate 10M output.mp4

4. Intelligent Constant Quality (ICQ)

If supported by your hardware driver (primarily Intel media drivers), ICQ offers a content-adaptive constant quality mode, which is more efficient than CQP.

• Flag: -rc_mode ICQ
• Quality Setting: Use -global_quality (ranges from 1 to 51, with lower values yielding higher quality).

Example:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i input.mp4 -c:v hevc_vaapi -rc_mode ICQ -global_quality 20 output.mp4

Adjusting Quality and Speed Presets

Apart from rate control, you can fine-tune the hardware encoder’s performance and compression efficiency.

Compression Level (Speed vs. Quality)

VAAPI uses the -compression_level option to set the trade-off between encoding speed and compression efficiency. * Range: Typically 1 to 7 (on Intel drivers). * Behavior: 1 represents the slowest speed but the highest quality/compression efficiency. 7 represents the fastest speed but lowest compression efficiency. The default is usually 4.

Example for maximum quality:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i input.mp4 -c:v hevc_vaapi -rc_mode VBR -b:v 5M -compression_level 1 output.mp4

Profile and Tier

You can enforce specific HEVC profiles (such as Main or Main 10 for 10-bit color) and tiers using the following options:

• Profile: -profile:v main or -profile:v main10
• Tier: -tier:v main or -tier:v high