How to Use FFmpeg vmaf_cuda for GPU Video Quality Evaluation

This guide covers how to set up and execute hardware-accelerated video quality metrics using the vmaf_cuda filter in FFmpeg. By offloading computational workloads from the CPU to an NVIDIA GPU, vmaf_cuda drastically reduces the processing time required to calculate Video Multi-Method Assessment Fusion (VMAF) scores. The following sections detail prerequisite requirements, pipeline architecture, and practical command-line examples.

Prerequisites and Requirements

To run vmaf_cuda, your environment must meet specific software and hardware criteria:

  1. NVIDIA GPU: A CUDA-compatible GPU (Pascal architecture or newer recommended) with up-to-date drivers.
  2. CUDA Toolkit: CUDA 11.0 or higher installed on the host system.
  3. libvmaf Built with CUDA: The underlying libvmaf library must be compiled with the -Denable_cuda=true flag using the Meson build system.
  4. FFmpeg with CUDA Filters: FFmpeg must be built with --enable-libvmaf, --enable-cuda-nvcc (or --enable-cuda-llvm), --enable-libnpp, and --enable-nonfree.

How the vmaf_cuda Pipeline Works

The vmaf_cuda filter requires frames to reside in CUDA device memory (AV_PIX_FMT_CUDA). Passing standard host system frames (yuv420p) directly into vmaf_cuda without transferring them to GPU memory will result in an error.

For optimal performance, the input streams should be decoded directly into CUDA memory using NVDEC, or uploaded to CUDA memory using the hwupload_cuda filter prior to reaching the evaluation step.

Standard Command Line Syntax

The most efficient pipeline keeps decoding, scaling, and scoring entirely on the GPU:

ffmpeg \
  -hwaccel cuda -hwaccel_output_format cuda -i distorted.mp4 \
  -hwaccel cuda -hwaccel_output_format cuda -i reference.mp4 \
  -filter_complex "[0:v][1:v]vmaf_cuda=model_path=/usr/local/share/model/vmaf_v0.6.1.json:log_fmt=json:log_path=vmaf_results.json" \
  -f null -

Breakdown of Options:

  • -hwaccel cuda -hwaccel_output_format cuda: Decodes the compressed video streams directly into NVIDIA VRAM, bypassing CPU bottlenecks.
  • -filter_complex "[0:v][1:v]vmaf_cuda=...": Directs both GPU-resident video frames into the vmaf_cuda filter. The distorted stream must be the first input [0:v], and the reference stream must be the second input [1:v].
  • model_path: Points to the VMAF model file (JSON or binary format).
  • log_fmt: Sets the output report format (json, xml, or csv).
  • log_path: Specifies the file path for the evaluation output.
  • -f null -: Discards video output so FFmpeg only processes metrics without encoding an output file.

Handling Mixed Inputs and Scaling

VMAF requires both the distorted and reference streams to have matching dimensions and pixel formats. If the two inputs differ in resolution, normalize them in CUDA memory using scale_npp or scale_cuda:

ffmpeg \
  -hwaccel cuda -hwaccel_output_format cuda -i distorted_720p.mp4 \
  -hwaccel cuda -hwaccel_output_format cuda -i reference_1080p.mp4 \
  -filter_complex "[0:v]scale_cuda=1920:1080[dist_scaled]; \
                   [dist_scaled][1:v]vmaf_cuda=model_path=/usr/local/share/model/vmaf_v0.6.1.json:log_fmt=json:log_path=vmaf_results.json" \
  -f null -

Uploading Software Decoded Video

If a codec is not supported by NVDEC hardware decoding, decode using the CPU and upload the frames to GPU memory inside the filter graph:

ffmpeg \
  -i distorted.mp4 \
  -i reference.mp4 \
  -filter_complex "[0:v]hwupload_cuda[dist_gpu]; \
                   [1:v]hwupload_cuda[ref_gpu]; \
                   [dist_gpu][ref_gpu]vmaf_cuda=model_path=/usr/local/share/model/vmaf_v0.6.1.json:log_fmt=json:log_path=vmaf_results.json" \
  -f null -

Verifying the Output

Once processing completes, FFmpeg outputs summary statistics in the terminal:

  • VMAF score: The final mean harmonic score across all processed frames.
  • Log file: Detailed per-frame metrics written to the file defined in log_path.