How to Use overlay_cuda in FFmpeg

This article provides a practical guide on how to use the overlay_cuda filter in FFmpeg to perform hardware-accelerated video overlaying using NVIDIA GPUs. You will learn the system requirements, the essential command-line syntax, and how to properly manage GPU memory and pixel formats to overlay videos or images onto a background video stream.

Prerequisites

To use overlay_cuda, your system must meet the following requirements: * An NVIDIA GPU that supports NVDEC and NVENC. * NVIDIA CUDA drivers installed. * A build of FFmpeg compiled with CUDA support (--enable-cuda, --enable-cuvid, --enable-nvenc, and --enable-libnpp).

Understanding the CUDA Overlay Pipeline

Unlike the standard CPU-based overlay filter, overlay_cuda requires all input frames to reside in GPU memory (CUDA hardware frames) before processing. If your inputs are in system memory (CPU), you must upload them to the GPU using the hwupload_cuda filter. If you decode the inputs using hardware acceleration (-hwaccel cuda), they are already in GPU memory.

Basic Syntax and Parameters

The basic syntax for the filter is:

overlay_cuda=x=X_POSITION:y=Y_POSITION

Example 1: Overlaying a PNG Image onto a Video

Since static PNG images cannot be decoded directly into CUDA memory via hardware accelerators, you must decode the PNG on the CPU, format it to include an alpha channel (for transparency), and upload it to the GPU.

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i main_video.mp4 \
-i logo.png \
-filter_complex "[1:v]format=rgba,hwupload_cuda[overlay];[0:v][overlay]overlay_cuda=x=10:y=10" \
-c:v h264_nvenc output.mp4

Command Breakdown: 1. -hwaccel cuda -hwaccel_output_format cuda -i main_video.mp4: Decodes the main video directly into CUDA device memory. 2. -i logo.png: Reads the static overlay image. 3. [1:v]format=rgba,hwupload_cuda[overlay]: Converts the PNG to RGBA format (to preserve transparency) and uploads it to the GPU. 4. [0:v][overlay]overlay_cuda=x=10:y=10: Overlays the uploaded image onto the hardware-decoded video at coordinates (10, 10). 5. -c:v h264_nvenc: Encodes the output video using the hardware-accelerated NVIDIA H.264 encoder.

Example 2: Overlaying Two Hardware-Decoded Videos

If you are overlaying two video files, both can be decoded directly on the GPU to maximize performance and avoid CPU-to-GPU transfer bottlenecks.

ffmpeg -hwaccel cuda -hwaccel_output_format cuda -i main_video.mp4 \
-hwaccel cuda -hwaccel_output_format cuda -i overlay_video.mp4 \
-filter_complex "[0:v][1:v]overlay_cuda=x=100:y=100" \
-c:v hevc_nvenc output.mp4

In this scenario, both input streams are decoded directly into CUDA memory. The overlay_cuda filter processes the frames entirely within the GPU, and the result is encoded using hevc_nvenc without any memory transfers back to the host system.