How to Use Sobel and Scharr Filters in FFmpeg

This guide explains how edge detection operators work in FFmpeg, specifically focusing on the edgedetect filter alongside the dedicated sobel and scharr filters. While the edgedetect filter relies on the Sobel operator internally to compute image gradients for Canny edge detection, FFmpeg also provides standalone filters to apply pure Sobel and Scharr gradient calculations directly to your video streams.

Understanding edgedetect and the Sobel Operator

The FFmpeg edgedetect filter implements the Canny edge detection algorithm. As part of this algorithm, it automatically applies a Sobel operator under the hood to calculate the intensity gradients of the image.

The edgedetect filter does not have a toggle to replace its internal Sobel kernel with a Scharr kernel, but you can control the thresholding and output visualization of the Sobel-driven edge detector using the following parameters:

  • low: Lower threshold for hysteresis (range 0 to 1, default 20/255).
  • high: Higher threshold for hysteresis (range 0 to 1, default 50/255).
  • mode: Edge drawing mode:
    • wires: White edges on a black background (default).
    • colormix: Overlays edges onto the original video colors.
    • canny: Standard black-and-white Canny output.

Example: Running edgedetect

ffmpeg -i input.mp4 -vf "edgedetect=low=0.1:high=0.4:mode=wires" -c:a copy output.mp4

Applying the Dedicated Sobel Filter

If you need the direct mathematical output of the first-order Sobel derivative without Canny hysteresis thresholding or non-maximum suppression, use FFmpeg's dedicated sobel filter instead of edgedetect.

  • scale: Multiplier for the filtered values (default 1).
  • delta: Value added to the filtered result (default 0).
  • planes: Flag determining which color planes to filter (default 15, which processes all planes).

Example: Applying Sobel Filtering

ffmpeg -i input.mp4 -vf "sobel=scale=1:delta=0" -c:a copy output.mp4

To run edge detection only on the luminance (luma) plane while leaving chroma untouched:

ffmpeg -i input.mp4 -vf "sobel=planes=1" -c:a copy output.mp4

Applying the Dedicated Scharr Filter

The Scharr operator provides better rotational symmetry and higher gradient accuracy than the Sobel operator for 3x3 kernels. FFmpeg provides a native scharr filter that matches the options of the sobel filter.

  • scale: Multiplier for the filtered values (default 1).
  • delta: Value added to the filtered result (default 0).
  • planes: Color planes to process (default 15).

Example: Applying Scharr Filtering

ffmpeg -i input.mp4 -vf "scharr=scale=1:delta=0" -c:a copy output.mp4

Applying Custom Sobel or Scharr Kernels via convolution

If you want custom directional edge detection (horizontal or vertical only) using Sobel or Scharr operators, you can use the generic 3x3 convolution filter.

Horizontal Sobel Kernel:

\[\begin{bmatrix} -1 & 0 & 1 \\ -2 & 0 & 2 \\ -1 & 0 & 1 \end{bmatrix}\]

ffmpeg -i input.mp4 -vf "convolution='-1 0 1 -2 0 2 -1 0 1':rdiv=1:bias=0" -c:a copy output.mp4

Horizontal Scharr Kernel:

\[\begin{bmatrix} -3 & 0 & 3 \\ -10 & 0 & 10 \\ -3 & 0 & 3 \end{bmatrix}\]

ffmpeg -i input.mp4 -vf "convolution='-3 0 3 -10 0 10 -3 0 3':rdiv=1:bias=0" -c:a copy output.mp4