Configure vp9_vaapi Rate Control and Quality in FFmpeg

This article provides a quick overview of how to configure the rate control and quality options for the Intel/AMD hardware-accelerated VP9 encoder (vp9_vaapi) in FFmpeg. You will learn how to set up Constant Quality (CQP), Variable Bitrate (VBR), and Constant Bitrate (CBR) modes, as well as how to fine-tune encoding speed and compression efficiency.

Setting Up VA-API in FFmpeg

Before applying rate control options, you must initialize the VA-API hardware device. A standard FFmpeg command template for hardware-accelerated VP9 encoding looks like this:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.mp4 -vf "hwupload" -c:v vp9_vaapi <rate_control_options> output.mkv

1. Constant Quality Mode (CQP)

In Constant Quality mode, the encoder maintains a consistent level of visual quality throughout the video, regardless of how much bitrate is required. This is the recommended mode for archival and general encoding.

For VA-API encoders, quality is controlled using the -global_quality flag.

• Option to use: -global_quality <value>
• Scale: Typically 1 to 255, where lower values mean better quality and higher bitrates.
• Recommended range: 100 to 130 for high-quality VP9 encodes.

Example Command:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.mp4 -vf "hwupload" -c:v vp9_vaapi -global_quality 110 output.mkv

2. Variable Bitrate Mode (VBR)

Variable Bitrate mode targets a specific average bitrate but allows the encoder to use more bits for complex scenes and fewer bits for simple scenes. This is ideal for streaming and file-size targeting.

To trigger VBR, specify a target bitrate with -b:v and set a maximum limit with -maxrate.

• Options to use: -b:v <target_bitrate> and -maxrate <max_bitrate>

Example Command:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.mp4 -vf "hwupload" -c:v vp9_vaapi -b:v 5M -maxrate 8M output.mkv

3. Constant Bitrate Mode (CBR)

Constant Bitrate mode forces the encoder to maintain a strict, unvarying bitrate. This mode is useful for strict bandwidth-limited live streaming scenarios.

To force CBR, set the target bitrate (-b:v), maximum bitrate (-maxrate), and buffer size (-bufsize) to the same value.

• Options to use: -b:v <bitrate> -maxrate <bitrate> -bufsize <bitrate>

Example Command:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.mp4 -vf "hwupload" -c:v vp9_vaapi -b:v 6M -maxrate 6M -bufsize 6M output.mkv

Adjusting the Quality and Speed Trade-off

You can control the encoder’s speed versus compression efficiency using the -compression_level option.

• Option to use: -compression_level <value>
• Scale: Typically 1 to 7 (depending on your hardware driver). Lower values prioritize speed (faster encoding, larger file size), while higher values prioritize compression efficiency (slower encoding, smaller file size).

Example Command (High Efficiency / Slower):

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i input.mp4 -vf "hwupload" -c:v vp9_vaapi -global_quality 110 -compression_level 7 output.mkv