FFmpeg Vulkan GPU Acceleration for Video Filtering

This guide provides a straightforward overview of how to leverage Vulkan-based hardware acceleration for video filtering in FFmpeg. You will learn the necessary prerequisites, how to initialize Vulkan hardware devices within your command line, and how to apply high-performance GPU-accelerated filters—such as scaling and format conversion—to drastically speed up your video processing workflows.

Prerequisites

To use Vulkan acceleration, your system must meet the following requirements: * Vulkan Drivers: Installed and updated graphics drivers (NVIDIA, AMD, or Intel) supporting Vulkan. * FFmpeg Compilation: A build of FFmpeg compiled with Vulkan support. Verify this by running ffmpeg -buildconf and checking for --enable-vulkan and --enable-libglslang.

The Basic Command Structure

Using Vulkan in FFmpeg requires initializing the GPU hardware device, uploading the video frames from system memory (CPU) to GPU memory, applying the Vulkan filter, and downloading the processed frames back to system memory for encoding.

The standard syntax is:

ffmpeg -init_hw_device vulkan=vk -filter_hw_device vk -i input.mp4 -vf "hwupload,scale_vulkan=1920:1080,hwdownload,format=yuv420p" output.mp4

Command Breakdown

• -init_hw_device vulkan=vk: Initializes a Vulkan hardware device named vk.
• -filter_hw_device vk: Tells FFmpeg’s filter graph to use the initialized vk device for hardware-accelerated filtering.
• hwupload: Uploads the input video frames from CPU RAM to Vulkan GPU memory.
• scale_vulkan=1920:1080: The Vulkan-native scaling filter, resizing the video to 1080p using GPU shaders.
• hwdownload: Downloads the processed frames from GPU memory back to CPU RAM.
• format=yuv420p: Converts the pixel format back to a standard format compatible with common software encoders (like libx264).

Useful Vulkan Filters

FFmpeg includes several built-in Vulkan filters. Here are the most common options:

1. Scaling (scale_vulkan)

Resizes video frames using the GPU. You can also specify the scaling algorithm.

-vf "hwupload,scale_vulkan=w=1280:h=720:scaler=bilinear,hwdownload,format=yuv420p"

2. Overlays (overlay_vulkan)

Blends two video streams together on the GPU, which is highly efficient for watermarks or picture-in-picture effects.

ffmpeg -init_hw_device vulkan=vk -filter_hw_device vk -i main.mp4 -i logo.png -filter_complex "[0:v]hwupload[main]; [1:v]hwupload[logo]; [main][logo]overlay_vulkan=x=10:y=10[out]; [out]hwdownload,format=yuv420p" output.mp4

3. Flip and Rotate (flip_vulkan)

Flips the video horizontally or vertically.

-vf "hwupload,flip_vulkan=horizontal=1,hwdownload,format=yuv420p"

Advanced: Full Hardware Pipeline (No CPU Bottleneck)

To achieve maximum performance, you can combine Vulkan filtering with hardware-accelerated decoding and encoding (such as NVIDIA NVDEC/NVENC or Intel QSV). This keeps the video frames entirely in GPU memory from start to finish.

Example using NVIDIA NVDEC, Vulkan filtering, and NVENC:

ffmpeg -hwaccel nvdec -hwaccel_output_format cuda -i input.mp4 -init_hw_device vulkan=vk -filter_hw_device vk -filter_complex "hwmap=derive_device=vulkan,scale_vulkan=1920:1080,hwmap=derive_device=cuda" -c:v h264_nvenc output.mp4

In this command: * hwaccel nvdec decodes the video directly onto the GPU. * hwmap=derive_device=vulkan maps the GPU frames from CUDA to Vulkan without copying them back to the CPU. * scale_vulkan processes the video. * hwmap=derive_device=cuda maps the frames back to CUDA for the h264_nvenc encoder.