How to Use FFmpeg scale_vulkan for Hardware Scaling

This guide provides a practical overview of utilizing the scale_vulkan filter in FFmpeg to perform high-performance, hardware-accelerated video scaling. You will learn the necessary prerequisites, the core command-line syntax, and how to construct commands to offload video resizing from your CPU to your GPU using the cross-platform Vulkan API.

Prerequisites

To use Vulkan acceleration in FFmpeg, your system must meet the following requirements: * Vulkan Drivers: Up-to-date graphics drivers for your GPU (NVIDIA, AMD, or Intel) with Vulkan support. * FFmpeg Build: An FFmpeg binary compiled with Vulkan support (compiled with --enable-vulkan and libshaderc or glslang).

Basic Syntax and Workflow

Vulkan processing in FFmpeg requires initializing a hardware device, uploading the video frames to GPU memory, applying the filter, and then downloading the frames back to system memory (unless you are using a GPU-based encoder).

The basic command structure is:

ffmpeg -init_hw_device vulkan=vk -i input.mp4 -filter_hw_device vk -vf "hwupload,scale_vulkan=1920:1080,hwdownload,format=yuv420p" output.mp4

Command Breakdown:

• -init_hw_device vulkan=vk: Initializes the Vulkan hardware device and names it vk.
• -filter_hw_device vk: Tells FFmpeg to associate the filter graph with the initialized Vulkan device.
• hwupload: Uploads the input video frames from system memory to Vulkan GPU memory.
• scale_vulkan=1920:1080: Resizes the video to 1920x1080 pixels on the GPU.
• hwdownload: Downloads the processed frames back to system memory.
• format=yuv420p: Converts the downloaded hardware frames back into a compatible software pixel format (like YUV 4:2:0) for standard software encoders (like libx264).

Adjusting Scaling Options

The scale_vulkan filter accepts several parameters to customize the output size and scaling algorithm.

Specifying Aspect Ratio

You can use mathematical expressions or maintain the aspect ratio using -1. For example, to scale to a width of 1280 while preserving the aspect ratio:

ffmpeg -init_hw_device vulkan=vk -i input.mp4 -filter_hw_device vk -vf "hwupload,scale_vulkan=1280:-1,hwdownload,format=yuv420p" output.mp4

Changing the Scaling Bilinear/Bicubic Filter

By default, the filter uses standard scaling, but you can explicitly specify the scaling scaler method or output format within the filter parameters:

scale_vulkan=w=1920:h=1080:scaler=bilinear

Advanced: Full GPU Pipeline (No HW Download)

To achieve maximum performance, you should keep the frames in GPU memory for both decoding, scaling, and encoding. This avoids the bottleneck of transferring frames between system RAM and VRAM.

If your GPU supports Vulkan-based decoding and encoding (or compatible VAAPI/NVDEC interfaces), you can chain them directly:

ffmpeg -init_hw_device vulkan=vk -hwaccel vulkan -hwaccel_output_format vulkan -i input.mp4 -filter_hw_device vk -vf "scale_vulkan=1920:1080" -c:v h264_nvenc output.mp4

(Note: Ensure your encoder, such as h264_nvenc or h264_vaapi, is compatible with the hardware frame output of your Vulkan filter).