How to Transcode Video with h264_nvenc in FFmpeg

This article provides a quick guide on how to use NVIDIA’s hardware-accelerated H.264 encoder, h264_nvenc, within FFmpeg. You will learn the system prerequisites, the basic transcoding command structure, how to control video quality and bitrate, and how to utilize NVENC presets for optimal performance and speed.

Prerequisites

To use NVIDIA hardware acceleration in FFmpeg, you must have: * An NVIDIA graphics card that supports NVENC (Kepler architecture or newer). * Up-to-date NVIDIA proprietary graphics drivers installed on your system. * A version of FFmpeg compiled with NVENC support (run ffmpeg -encoders | grep nvenc in your terminal to verify h264_nvenc is available).

Basic Transcoding Command

The simplest command to transcode an input video to H.264 using NVENC is:

ffmpeg -i input.mp4 -c:v h264_nvenc output.mp4

In this command: * -i input.mp4 specifies the source video file. * -c:v h264_nvenc selects the NVIDIA NVENC H.264 encoder for the video stream. * output.mp4 is the resulting transcoded file.

Controlling Quality and Bitrate

Unlike CPU-based encoders like libx264 which use Constant Rate Factor (-crf), h264_nvenc uses different rate control methods.

1. Constant Quality (CQP)

This is the recommended mode for one-pass file-based transcoding where file size is not a strict constraint. Lower values yield better quality but larger file sizes (a range of 19 to 23 is typical for high quality).

ffmpeg -i input.mp4 -c:v h264_nvenc -rc constqp -qp 23 output.mp4

2. Target Bitrate (VBR / CBR)

To target a specific average bitrate, use the -b:v flag. You can also set a maximum bitrate (-maxrate) and buffer size (-bufsize) for streaming or device compatibility.

ffmpeg -i input.mp4 -c:v h264_nvenc -b:v 5M -maxrate:v 8M -bufsize:v 8M output.mp4

Using NVENC Presets and Profiles

NVENC uses presets to balance encoding speed and output quality. Newer FFmpeg versions use presets ranging from p1 (fastest, lowest quality) to p7 (slowest, highest quality). The default is p4.

ffmpeg -i input.mp4 -c:v h264_nvenc -preset p6 output.mp4

(Note: Older FFmpeg builds may use legacy presets like slow, medium, or fast.)

To ensure maximum compatibility with older playback devices, you can limit the H.264 profile to main or high:

ffmpeg -i input.mp4 -c:v h264_nvenc -profile:v high output.mp4

Full GPU Pipeline (Hardware Decoding and Encoding)

To maximize performance, you can decode the input video using the GPU before encoding it. This minimizes CPU usage and avoids data transfer bottlenecks between host memory and GPU memory.

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -c:v h264_nvenc output.mp4