Configure hevc_nvenc Rate Control and Quality in FFmpeg

This article provides a straightforward guide on how to configure the rate control and quality options for the NVIDIA NVENC HEVC (hevc_nvenc) hardware encoder in FFmpeg. You will learn how to select and configure different rate control modes—such as Constant QP, VBR, and CBR—and how to utilize quality-enhancing settings like presets, lookahead, and adaptive quantization to optimize your video encodes.

Rate Control Modes

Rate control determines how the encoder allocates data (bitrate) across the video frames. The hevc_nvenc encoder supports several rate control modes using the -rc option, combined with target bitrate (-b:v) and quality parameters.

1. Constant QP (CQP)

Constant QP defines a fixed quantization parameter for all frames, ensuring consistent image quality at the expense of unpredictable file sizes. It is ideal for local archiving where disk space is not a primary concern.

• -rc constqp: Enables Constant QP mode.
• -qp: Sets the global QP value (typically between 0 and 51, where lower means better quality). You can also specify different QP values for I, P, and B frames using -init_qp_i, -init_qp_p, and -init_qp_b.

ffmpeg -i input.mp4 -c:v hevc_nvenc -rc constqp -qp 23 output.mp4

2. Variable Bitrate (VBR)

VBR allows the bitrate to fluctuate based on the complexity of the scene. It is suitable for streaming and general encoding.

• -rc vbr: Enables standard Variable Bitrate.
• -b:v: Sets the target average bitrate.
• -maxrate: Sets the maximum peak bitrate.

ffmpeg -i input.mp4 -c:v hevc_nvenc -rc vbr -b:v 5M -maxrate 8M -bufsize 10M output.mp4

3. Constrained Quality (CQ / Target Quality VBR)

This mode acts like a VBR mode but prioritizes a specific quality level (defined by the CQ parameter) while respecting a maximum bitrate limit if specified.

• -rc vbr: Must be set to VBR.
• -cq: Sets the target quality level (lower values equal higher quality, usually 19 to 24 is a good range).
• -b:v 0: Ensures the encoder prioritizes the CQ target over a specific average bitrate.

ffmpeg -i input.mp4 -c:v hevc_nvenc -rc vbr -cq 22 -b:v 0 output.mp4

4. Constant Bitrate (CBR)

CBR maintains a strict, constant bitrate throughout the entire video. This is essential for live streaming environments with fixed bandwidth constraints.

• -rc cbr: Enables Constant Bitrate.
• -b:v: Sets the target bitrate.
• -maxrate and -bufsize: Must match the target bitrate and vbv buffer size to enforce a strict constant stream.

ffmpeg -i input.mp4 -c:v hevc_nvenc -rc cbr -b:v 4M -maxrate 4M -bufsize 8M output.mp4

Quality and Performance Tuning Options

Beyond rate control, several settings directly impact the compression efficiency and visual quality of the output video.

Presets and Tuning

NVIDIA’s newer SDKs use a preset system ranging from p1 (fastest/lowest quality) to p7 (slowest/highest quality).

• -preset: Choose from p1 to p7. For older drivers, legacy presets like slow, medium, and fast are still supported.
• -tune: Optimizes the encoder for specific use cases. Common options include hq (High Quality), ll (Low Latency), and ull (Ultra-Low Latency).

ffmpeg -i input.mp4 -c:v hevc_nvenc -preset p6 -tune hq output.mp4

Lookahead and Adaptive Quantization

These features analyze adjacent frames and frame regions to distribute bits more efficiently to areas where human eyes notice quality degradation the most.

• -rc-lookahead: Enables temporal lookahead, allowing the encoder to scan ahead (e.g., 20 frames) to optimize frame types and bitrate distribution.
• -spatial-aq 1: Enables Spatial Adaptive Quantization, which adjusts QP within a frame to keep flat areas looking clean and textured areas detailed.
• -temporal-aq 1: Enables Temporal Adaptive Quantization, which prevents pulsing quality artifacts in fast-moving scenes.

ffmpeg -i input.mp4 -c:v hevc_nvenc -rc vbr -b:v 6M -preset p6 -rc-lookahead 20 -spatial-aq 1 -temporal-aq 1 output.mp4