Custom Morphological Grids with FFmpeg Neighbor Filters

This article explains how to use FFmpeg’s neighbor-based morphological filters—specifically erosion and dilation—to apply custom grid operations to video frames. You will learn how the coordinates parameter acts as a bitmask to define custom 3x3 structuring elements (grids) and how to control the intensity of these operations using thresholds.

Understanding FFmpeg’s Neighbor Filters

In FFmpeg, morphological operations are performed using the erosion and dilation filters (under the hood, these are implemented as “neighbor” filters).

• Erosion replaces the center pixel’s value with the minimum value found among its designated neighbors. This has the effect of shrinking bright areas and expanding dark areas.
• Dilation replaces the center pixel’s value with the maximum value found among its designated neighbors. This expands bright areas and shrinks dark areas.

By default, these filters look at all eight surrounding pixels in a 3x3 grid. However, you can customize this grid to create specific shapes (like lines or crosses) using the coordinates parameter.

The 3x3 Grid Coordinate Bitmask

The coordinates parameter accepts an integer from 0 to 255. This integer is an 8-bit bitmask where each bit corresponds to one of the eight neighboring pixels around the center pixel.

The binary mapping for the 3x3 grid is structured as follows:

To target a custom grid shape, you add the decimal values of the positions you want the filter to consider.

Practical Examples of Custom Grids

1. Cross Shape (Vertical and Horizontal)

To restrict the filter to only look at the immediate top, bottom, left, and right pixels, add their values: * Top (2) + Left (8) + Right (16) + Bottom (64) = 90

FFmpeg Command:

ffmpeg -i input.mp4 -vf "erosion=coordinates=90" output.mp4

2. Horizontal Only

To apply the morphological operation purely along the horizontal axis, select only the left and right neighbors: * Left (8) + Right (16) = 24

FFmpeg Command:

ffmpeg -i input.mp4 -vf "dilation=coordinates=24" output.mp4

3. Vertical Only

To apply the operation purely along the vertical axis, select only the top and bottom neighbors: * Top (2) + Bottom (64) = 66

FFmpeg Command:

ffmpeg -i input.mp4 -vf "erosion=coordinates=66" output.mp4

4. Diagonal (Top-Left to Bottom-Right)

To apply the operation along a diagonal axis: * Top-Left (1) + Bottom-Right (128) = 129

FFmpeg Command:

ffmpeg -i input.mp4 -vf "dilation=coordinates=129" output.mp4

Fine-Tuning with Thresholds

By default, the filters will replace the pixel value completely with the min or max neighboring value. You can limit this behavior using the threshold parameters: threshold0, threshold1, threshold2, and threshold3 (corresponding to the Y, U, V, and Alpha planes respectively, or R, G, B, A).

The threshold limits the maximum allowed difference between the original pixel value and the new morphed value.

Example Command (limiting Y-plane change to 10 units):

ffmpeg -i input.mp4 -vf "dilation=coordinates=90:threshold0=10" output.mp4

This restricts the dilation effect on the brightness (Luma) channel, preventing extreme shifts and producing a subtler morphological effect.