Using FFmpeg overlay_opencl for Hardware Video Overlay

The overlay_opencl filter in FFmpeg enables hardware-accelerated video compositing by offloading the task of overlaying one video stream or image onto another to an OpenCL-capable GPU. This process significantly reduces CPU utilization and can accelerate rendering workflows, especially when processing high-resolution content. To use this filter successfully, you must initialize an OpenCL device context, transfer your video frames into the OpenCL memory space using hardware upload filters, apply the overlay, and then retrieve or encode the resulting hardware frames.

Prerequisites

Your FFmpeg build must be compiled with OpenCL support. You can verify this by running:

ffmpeg -filters | grep opencl

Ensure that overlay_opencl appears in the list and that your system has appropriate OpenCL runtime drivers installed for your GPU (Intel, AMD, or NVIDIA).

Basic Command Syntax

Using overlay_opencl requires setting up an OpenCL hardware device and converting software frames into hardware frames using hwupload.

Here is a standard example overlaying a logo onto a background video:

ffmpeg -init_hw_device opencl=gpu:0.0 -filter_hw_device gpu \
-i main_video.mp4 -i watermark.png \
-filter_complex "\
[0:v]format=yuv420p,hwupload[base]; \
[1:v]format=yuva420p,hwupload[overlay]; \
[base][overlay]overlay_opencl=x=50:y=50[out]; \
[out]hwdownload,format=yuv420p" \
-c:v libx264 -c:a copy output.mp4

Breakdown of the Pipeline

  1. -init_hw_device opencl=gpu:0.0: Initializes an OpenCL device named gpu referencing platform 0, device 0.
  2. -filter_hw_device gpu: Sets the default hardware device context for the filtergraph.
  3. format=yuv420p,hwupload: Converts the incoming software frames to a compatible pixel format and uploads them to OpenCL memory space. Transparent overlays typically require a format supporting an alpha channel, such as yuva420p.
  4. overlay_opencl=x=50:y=50: Composites the overlay stream onto the base video stream starting at pixel coordinates X=50, Y=50 inside GPU memory.
  5. hwdownload,format=yuv420p: Retrieves the composited frames from GPU memory back to system memory so a software encoder like libx264 can process them.

Common Parameters

The overlay_opencl filter accepts several configuration options:

  • x: Sets the horizontal offset of the overlay. Can be an integer or an expression. Default is 0.
  • y: Sets the vertical offset of the overlay. Can be an integer or an expression. Default is 0.
  • eof_action: Determines what happens when the secondary (overlay) stream reaches its end:
    • assert: Terminate the program with an error if the secondary stream ends before the primary.
    • repeat: Repeat the last frame of the overlay stream indefinitely (default).
    • endall: End both streams when the shortest input ends.
    • pass: Pass the main input stream through unaltered after the overlay ends.

Pure Hardware Pipeline Example

To maximize performance, you can keep the entire pipeline on the GPU by pairing overlay_opencl with a hardware-accelerated encoder (such as VAAPI on Linux):

ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 \
-init_hw_device opencl=ocl@va \
-filter_hw_device ocl \
-hwaccel vaapi -hwaccel_output_format vaapi -i main_video.mp4 \
-i watermark.png \
-filter_complex "\
[0:v]hwmap=derive_device=opencl[base]; \
[1:v]format=yuva420p,hwupload[overlay]; \
[base][overlay]overlay_opencl=x=W-w-10:y=H-h-10[out]; \
[out]hwmap=derive_device=vaapi:reverse=1[vaout]" \
-map "[vaout]" -c:v h264_vaapi output.mp4

This configuration avoids transferring decoded frames back and forth between system RAM and GPU VRAM, minimizing pipeline latency and maximizing throughput.