How to Use the FFmpeg overlay_opencl Filter

This article provides a quick guide on how to use the overlay_opencl filter in FFmpeg to overlay one video or image onto another using GPU acceleration. You will learn the necessary OpenCL hardware initialization steps, how to upload video frames to the GPU, how to apply the overlay filter with custom coordinates, and how to download the final output back for encoding.

To use OpenCL filters in FFmpeg, you cannot simply pass standard software video streams directly to the filter. You must initialize your OpenCL hardware device, upload the input streams to GPU memory, apply the OpenCL filter, and then download the results back to system memory.

The Basic Command Structure

Here is a complete command-line example that overlays a secondary video (or image) onto a main video at specific coordinates (X=10, Y=10) using OpenCL:

ffmpeg -init_hw_device opencl=gpu -filter_hw_device gpu \
-i main_video.mp4 -i overlay_image.png \
-filter_complex "[0:v]format=nv12,hwupload[main]; [1:v]format=nv12,hwupload[over]; [main][over]overlay_opencl=x=10:y=10[out]; [out]hwdownload,format=nv12" \
-c:v libx264 -an output.mp4

Command Breakdown

• -init_hw_device opencl=gpu: This initializes the OpenCL hardware device and aliases it as gpu. FFmpeg will attempt to use your system’s primary GPU for OpenCL processing.
• -filter_hw_device gpu: This instructs FFmpeg to use the newly initialized gpu device for any hardware filters defined in the filtergraph.
• format=nv12,hwupload: Before applying OpenCL filters, the software frames must be converted to a compatible pixel format (such as nv12 or yuv420p) and then sent to the GPU using the hwupload filter. This must be done for both the main input ([0:v]) and the overlay input ([1:v]).
• overlay_opencl=x=10:y=10: This is the core filter. It takes the two uploaded GPU frames and overlays the second one onto the first. The parameters x and y specify the top-left offset coordinates in pixels for the overlay.
• hwdownload,format=nv12: Once the overlay is processed on the GPU, the hwdownload filter pulls the frames back into system memory. The subsequent format filter matches the pixel format back to a software-compatible format so that standard encoders (like libx264) can process it.

Supported Parameters

The overlay_opencl filter supports several parameters to control positioning:

• x: The horizontal position of the overlay. It can be a static pixel number or an expression (e.g., main_w-overlay_w-10 to place it 10 pixels from the right edge).
• y: The vertical position of the overlay. It can also be a static number or an expression (e.g., main_h-overlay_h-10 to place it 10 pixels from the bottom edge).

By utilizing these steps, you can offload the CPU-intensive video overlay process directly to your graphics hardware, significantly increasing rendering speeds.