How to Write a Custom FFmpeg Video Filter

This guide walks you through developing and integrating a custom video filter into FFmpeg’s libavfilter library. You will learn the structural requirements of an AVFilter, how to implement core callbacks for configuration and pixel processing, and the exact files you must modify within the FFmpeg source tree to register, compile, and run your new filter.

1. Set Up the Source Code

Custom internal filters must be compiled alongside FFmpeg. Clone the source repository to begin:

git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg

2. Create the Filter Implementation

Create a new C source file in the libavfilter/ directory, for example, libavfilter/vf_invert_brightness.c.

A functional video filter requires:

  • A private context struct to store filter options and internal state.
  • AVOption definitions for user-configurable parameters.
  • Callback functions for negotiating formats, configuring links, and processing video frames.
  • Input and output pad definitions (AVFilterPad).
  • The main AVFilter structure definition.

Below is a complete, minimal implementation of a custom filter that modifies pixel luma:

#include "libavutil/opt.h"
#include "libavutil/imgutils.h"
#include "avfilter.h"
#include "internal.h"
#include "video.h"

typedef struct CustomContext {
    const AVClass *class;
    int invert;
} CustomContext;

#define OFFSET(x) offsetof(CustomContext, x)
#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM

static const AVOption custom_options[] = {
    { "invert", "Invert luma values", OFFSET(invert), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, FLAGS },
    { NULL }
};

AVFILTER_DEFINE_CLASS(custom);

static int filter_frame(AVFilterLink *inlink, AVFrame *in)
{
    AVFilterContext *ctx = inlink->dst;
    CustomContext *s = ctx->priv;
    AVFilterLink *outlink = ctx->outputs[0];
    AVFrame *out;
    int x, y;

    if (av_frame_is_writable(in)) {
        out = in;
    } else {
        out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
        if (!out) {
            av_frame_free(&in);
            return AVERROR(ENOMEM);
        }
        av_frame_copy_props(out, in);
    }

    /* Process the Y (luma) plane for planar formats like YUV420P */
    for (y = 0; y < in->height; y++) {
        uint8_t *src = in->data[0] + y * in->linesize[0];
        uint8_t *dst = out->data[0] + y * out->linesize[0];

        for (x = 0; x < in->width; x++) {
            dst[x] = s->invert ? (255 - src[x]) : src[x];
        }
    }

    if (in != out)
        av_frame_free(&in);

    return ff_filter_frame(outlink, out);
}

static const AVFilterPad custom_inputs[] = {
    {
        .name         = "default",
        .type         = AVMEDIA_TYPE_VIDEO,
        .filter_frame = filter_frame,
    },
};

static const AVFilterPad custom_outputs[] = {
    {
        .name = "default",
        .type = AVMEDIA_TYPE_VIDEO,
    },
};

const AVFilter ff_vf_custom = {
    .name          = "custom",
    .description   = NULL_IF_CONFIG_SMALL("Applies custom luma inversion."),
    .priv_size     = sizeof(CustomContext),
    .priv_class    = &custom_class,
    FILTER_INPUTS(custom_inputs),
    FILTER_OUTPUTS(custom_outputs),
    FILTER_PIXFMTS(AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV444P, AV_PIX_FMT_GRAY8),
    .flags         = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC,
};

3. Register the Filter in the Build System

FFmpeg discovers and builds filters based on static declarations and make rules. You must edit two files inside libavfilter/:

A. Edit libavfilter/allfilters.c

Declare your filter instance so the registration system recognizes it. Add the external declaration alphabetically among the video filters:

extern const AVFilter ff_vf_custom;

B. Edit libavfilter/Makefile

Add the object file mapping so make compiles your code when the filter is enabled:

OBJS-$(CONFIG_CUSTOM_FILTER) += vf_custom.o

4. Compile and Validate

Run the configuration script and build the project. The build system will detect your filter and generate CONFIG_CUSTOM_FILTER=yes in config.mak.

./configure --disable-shared --enable-pthreads
make -j$(nproc)

Verify that the filter is properly registered with libavfilter:

./ffmpeg -filters | grep custom

You should see output similar to:

 ... custom            V->V       Applies custom luma inversion.

5. Test the Filter

Run your custom filter on a test video to verify that frames are processed without errors or memory leaks:

./ffmpeg -i input.mp4 -vf "custom=invert=1" -c:a copy output.mp4