FFmpeg VP9 Hardware Transcoding with NVDEC on Linux

This guide provides a straightforward walk-through on how to leverage NVIDIA hardware acceleration (NVDEC) to decode VP9 video files and transcode them efficiently using FFmpeg on a Linux system. You will learn the necessary system prerequisites, the exact FFmpeg command structures to initiate the hardware-accelerated pipeline, and how to verify that your GPU is handling the decoding workload.

Prerequisites

Before starting, ensure your system meets the following requirements:

  • NVIDIA GPU: A graphics card that supports VP9 hardware decoding (GeForce GTX 950/960 with GM206, or any Pascal architecture and newer GPUs).

  • NVIDIA Drivers: Proprietary NVIDIA drivers installed and configured.

  • FFmpeg with CUDA support: FFmpeg must be compiled with NVDEC/NVENC support. You can verify this by running:

    ffmpeg -decoders | grep vp9

    Look for vp9_cuvid (NVIDIA CUVID/NVDEC VP9 decoder) in the output.

The Full Hardware Transcoding Command

To decode a VP9 video using NVDEC and transcode it to H.264 using NVENC (keeping the entire pipeline on the GPU to avoid CPU bottlenecks), use the following command:

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -c:v vp9_cuvid -i input.webm -c:v h264_nvenc -preset p4 -b:v 5M output.mp4

Command Breakdown:

  • -hwaccel cuda: Enables CUDA hardware acceleration for the transcoding pipeline.
  • -hwaccel_output_format cuda: Keeps the decoded video frames in GPU VRAM to prevent slow copying back to system memory (RAM).
  • -c:v vp9_cuvid: Explicitly specifies the NVDEC hardware-accelerated VP9 decoder.
  • -i input.webm: The input VP9 video file.
  • -c:v h264_nvenc: Specifies the NVIDIA hardware-accelerated H.264 encoder for the output file (you can also use hevc_nvenc for H.265/HEVC).
  • -preset p4: Sets the NVENC encoding preset (ranging from p1 for fastest to p7 for slowest/best quality).
  • -b:v 5M: Sets the target video bitrate to 5 Mbps.

Decoding with GPU and Encoding with CPU

If you want to decode using the GPU but encode using a software/CPU-based encoder like x264, you must omit the CUDA hardware output format flag to copy the frames back to system memory:

ffmpeg -hwaccel cuda -c:v vp9_cuvid -i input.webm -c:v libx264 -crf 23 output.mp4

Verifying Hardware Utilization

To confirm that your GPU is actively decoding the VP9 video, open a separate terminal window while the FFmpeg command is running and execute:

nvidia-smi

In the output, verify that ffmpeg is listed under the "Processes" section, and check that the "DEC" (Decoder) utilization percentage is active.