How to Use overlay_opencl Filter in FFmpeg

Using hardware acceleration in FFmpeg can drastically reduce video processing times by offloading intensive tasks to the GPU. This article explains how to use the overlay_opencl filter in FFmpeg to overlay one video or image onto another using OpenCL, covering device initialization, filtergraph syntax, and a practical command-line example.

Understanding OpenCL in FFmpeg

To use OpenCL filters like overlay_opencl, you cannot simply pass standard video inputs directly to the filter. Because OpenCL operates on the GPU, you must first initialize an OpenCL hardware device, upload your video frames from system memory (RAM) to graphics memory (VRAM), apply the filter, and then download the results back to system memory for encoding.

Step-by-Step Command Structure

Here is a practical command template to overlay an image onto a video using OpenCL:

ffmpeg -init_hw_device opencl=gpu -filter_hw_device gpu \
-i main_video.mp4 -i overlay_image.png \
-filter_complex "[0:v]format=nv12,hwupload[main]; [1:v]format=rgba,hwupload[overlay]; [main][overlay]overlay_opencl=x=10:y=10[temp]; [temp]hwdownload,format=nv12" \
-c:v libx264 -c:a copy output.mp4

Breakdown of the Command

• -init_hw_device opencl=gpu: This initializes the OpenCL hardware device and labels it as gpu.
• -filter_hw_device gpu: This instructs FFmpeg to use the newly created gpu device for all hardware-accelerated filters in the filtergraph.
• format=nv12,hwupload / format=rgba,hwupload: Before sending frames to overlay_opencl, they must be converted to a compatible pixel format and uploaded to the GPU using the hwupload filter.
• overlay_opencl=x=10:y=10: This is the core OpenCL filter. It places the overlay input onto the main input at the specified coordinates (x=10, y=10).
• hwdownload,format=nv12: After the overlay is processed on the GPU, the video frames must be downloaded back to the CPU memory and formatted (typically to nv12 or yuv420p) so standard software encoders like libx264 can read them.

Supported Arguments for overlay_opencl

The overlay_opencl filter supports the standard placement parameters:

• x: The horizontal position of the overlay (defaults to 0).
• y: The vertical position of the overlay (defaults to 0).

You can also use expressions for dynamic positioning, such as placing the overlay in the center of the screen:

overlay_opencl=x=(W-w)/2:y=(H-h)/2

(Where W and H are the width and height of the main video, and w and h are the width and height of the overlay element).