Feed Frames to FFmpeg Filtergraph Using Buffer Source

In the FFmpeg C API, feeding raw decoded audio or video data into a filtergraph requires using the buffer (for video) or abuffer (for audio) source filter. This article provides a direct guide on initializing these buffer sources, passing AVFrame instances into the filter pipeline using av_buffersrc_add_frame_flags(), flushing the graph at stream termination, and pulling the filtered output from the sink.

1. Filtergraph Architecture

An FFmpeg filtergraph requires an entry point and an exit point:

Source: buffer (video) or abuffer (audio) receives AVFrame objects from your application.
Sink: buffersink (video) or abuffersink (audio) outputs processed AVFrame objects.

[ Your App: AVFrame ] 
         │
         ▼
[ buffer / abuffer ] ──► [ Filter Chain ] ──► [ buffersink / abuffersink ]
                                                       │
                                                       ▼
                                            [ Filtered AVFrame ]

2. Initializing the Buffer Source

The buffer source must be initialized with parameters matching the input frames (such as dimensions, pixel format, time base, or audio sample rate).

Video (`buffer`) Configuration

#include <libavfilter/avfilter.h>
#include <libavfilter/buffersrc.h>
#include <libavfilter/buffersink.h>
#include <libavutil/opt.h>

AVFilterGraph *filter_graph = avfilter_graph_alloc();
const AVFilter *buffersrc = avfilter_get_by_name("buffer");
const AVFilter *buffersink = avfilter_get_by_name("buffersink");
AVFilterContext *buffersrc_ctx;
AVFilterContext *buffersink_ctx;

char args[512];
snprintf(args, sizeof(args),
         "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
         width, height, pix_fmt,
         time_base_num, time_base_den,
         aspect_num, aspect_den);

int ret = avfilter_graph_create_filter(&buffersrc_ctx, buffersrc, "in",
                                       args, NULL, filter_graph);
if (ret < 0) {
    // Handle error
}

Audio (`abuffer`) Configuration

const AVFilter *abuffersrc = avfilter_get_by_name("abuffer");
AVFilterContext *abuffersrc_ctx;

char ch_layout_str[64];
av_channel_layout_describe(&channel_layout, ch_layout_str, sizeof(ch_layout_str));

char args[512];
snprintf(args, sizeof(args),
         "sample_rate=%d:sample_fmt=%s:channel_layout=%s:time_base=%d/%d",
         sample_rate, av_get_sample_fmt_name(sample_fmt),
         ch_layout_str, time_base_num, time_base_den);

ret = avfilter_graph_create_filter(&abuffersrc_ctx, abuffersrc, "in",
                                   args, NULL, filter_graph);

3. Linking the Graph and Finalizing

Create your sink filter, connect your inputs to outputs (either directly or via a filter string parsed by avfilter_graph_parse_ptr()), and configure the graph.

// Create sink
ret = avfilter_graph_create_filter(&buffersink_ctx, buffersink, "out",
                                   NULL, NULL, filter_graph);

// Link source directly to sink (or place intermediate filters in between)
ret = avfilter_link(buffersrc_ctx, 0, buffersink_ctx, 0);

// Validate and configure the graph
ret = avfilter_graph_config(filter_graph, NULL);

4. Pushing Frames into the Buffer Source

To feed frames into the initialized filtergraph, call av_buffersrc_add_frame_flags().

// Push a decoded frame into the filtergraph
int ret = av_buffersrc_add_frame_flags(buffersrc_ctx, input_frame, AV_BUFFERSRC_FLAG_KEEP_REF);
if (ret < 0) {
    // Handle buffer source feed error
}

AV_BUFFERSRC_FLAG_KEEP_REF: Keeps the reference to input_frame intact in your code by taking a new reference internally. If omitted, the filtergraph takes ownership of input_frame and unreferences it.
Flushing: When the input stream ends, send NULL as the frame pointer to signal End Of File (EOF):
```
av_buffersrc_add_frame_flags(buffersrc_ctx, NULL, 0);
```

5. Pulling Processed Frames from the Sink

After feeding a frame, retrieve all available filtered frames in a loop until AVERROR(EAGAIN) or AVERROR_EOF is returned:

AVFrame *filtered_frame = av_frame_alloc();

while (1) {
    int ret = av_buffersink_get_frame(buffersink_ctx, filtered_frame);
    if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
        break; // Need more input frames or EOF reached
    }
    if (ret < 0) {
        // Handle filter processing error
        break;
    }

    // Process the filtered frame (e.g., encode or display)

    av_frame_unref(filtered_frame);
}

av_frame_free(&filtered_frame);

6. Cleanup

Free the entire graph and its associated filter contexts when processing is complete:

avfilter_graph_free(&filter_graph);