How to Use FFmpeg scale_vulkan for Video Scaling

This guide provides a practical walkthrough on using the scale_vulkan filter in FFmpeg for GPU-accelerated video resizing. It covers the required hardware device initialization, how to upload and download frames between system memory and the GPU, and working command-line examples for scaling videos efficiently using the Vulkan API.

Prerequisites

To use scale_vulkan, your FFmpeg build must be compiled with Vulkan support (--enable-vulkan). You must also have a Vulkan-compatible GPU and up-to-date graphics drivers installed on your system.

Understanding the Vulkan Filter Pipeline

Vulkan filters in FFmpeg do not operate directly on standard software pixel formats. Instead, processing requires a specific pipeline:

  1. Initialize the hardware device: Define a Vulkan device context.
  2. Upload frames to the GPU: Convert frames to a hardware-compatible format using hwupload.
  3. Apply the filter: Process the frames using scale_vulkan.
  4. Download or encode: Retrieve the frames back to system memory with hwdownload for software encoding, or pass them to a hardware encoder.

Basic Command Example

The following command resizes an input video to 1920x1080 using the scale_vulkan filter and encodes the output with libx264:

ffmpeg -init_hw_device vulkan=vk:0 -filter_hw_device vk -i input.mp4 \
  -vf "format=yuv420p,hwupload,scale_vulkan=w=1920:h=1080,hwdownload,format=yuv420p" \
  -c:v libx264 -c:a copy output.mp4

Breakdown of Arguments

  • -init_hw_device vulkan=vk:0: Initializes a Vulkan device named vk on GPU index 0.
  • -filter_hw_device vk: Sets vk as the default hardware device for the filter graph.
  • format=yuv420p: Ensures the input pixel format is compatible before uploading.
  • hwupload: Moves the frames from CPU memory to Vulkan GPU memory.
  • scale_vulkan=w=1920:h=1080: Scales the video on the GPU to a width of 1920 and a height of 1080.
  • hwdownload: Transfers the processed frames from GPU memory back to CPU memory.
  • format=yuv420p: Restores the software pixel format required by the output encoder.

Maintaining Aspect Ratio

To scale one dimension while keeping the original aspect ratio, set the other dimension to -1 or -2 (ensuring the dimension is divisible by 2):

ffmpeg -init_hw_device vulkan=vk:0 -filter_hw_device vk -i input.mp4 \
  -vf "format=yuv420p,hwupload,scale_vulkan=w=1280:h=-2,hwdownload,format=yuv420p" \
  -c:v libx264 -c:a copy output.mp4

Scaling Algorithms

The scale_vulkan filter supports custom scaling algorithms via the scaler parameter. Available options include bilinear (default) and nearest.

Example specifying nearest-neighbor scaling:

ffmpeg -init_hw_device vulkan=vk:0 -filter_hw_device vk -i input.mp4 \
  -vf "format=yuv420p,hwupload,scale_vulkan=w=1920:h=1080:scaler=nearest,hwdownload,format=yuv420p" \
  -c:v libx264 -c:a copy output.mp4