Feed Frames to FFmpeg Filtergraph with Buffer Source

This article explains how to input raw video or audio frames into an FFmpeg filtergraph using the C API. You will learn how to configure the buffer (for video) or abuffer (for audio) source filters, initialize the filtergraph, and programmatically push AVFrame structures into the graph for processing.

1. Initialize the Filtergraph and Buffer Source

To feed frames into a filtergraph, you must first create an AVFilterGraph and instantiate a buffer source filter. The buffer source acts as the entry point for your raw frames.

You need to pass configuration parameters as a string to the buffer source filter during initialization. For video, this includes the resolution, pixel format, timebase, and sample aspect ratio.

#include <libavfilter/avfilter.h>
#include <libavfilter/buffersrc.h>
#include <libavutil/opt.h>

AVFilterGraph *filter_graph = avfilter_graph_alloc();
AVFilterContext *buffersrc_ctx = NULL;
const AVFilter *buffersrc = avfilter_get_by_name("buffer"); // Use "abuffer" for audio

char args[512];
int width = 1920;
int height = 1080;
enum AVPixelFormat pix_fmt = AV_PIX_FMT_YUV420P;
AVRational time_base = {1, 25};
AVRational pixel_aspect = {1, 1};

// Format the configuration arguments string
snprintf(args, sizeof(args),
         "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
         width, height, pix_fmt, 
         time_base.num, time_base.den, 
         pixel_aspect.num, pixel_aspect.den);

// Create the buffer source filter context and add it to the graph
int ret = avfilter_graph_create_filter(&buffersrc_ctx, buffersrc, "in",
                                       args, NULL, filter_graph);
if (ret < 0) {
    // Handle error
}

For audio (abuffer), the arguments string format requires sample rate, sample format, channel layout (or channel count), and timebase:

// Example args for audio:
snprintf(args, sizeof(args),
         "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%llx",
         time_base.num, time_base.den, sample_rate,
         av_get_sample_fmt_name(sample_fmt), channel_layout);

2. Connect and Configure the Graph

After setting up the source (buffer) and your destination (buffersink), you must link them together (either manually or using avfilter_graph_parse_ptr) and finalize the graph configuration.

// Always call this after building the filter pipeline to validate connections
ret = avfilter_graph_config(filter_graph, NULL);
if (ret < 0) {
    // Handle configuration error
}

3. Push Frames to the Buffer Source

Once the graph is configured, you can feed decoded or generated AVFrame objects into the filtergraph using av_buffersrc_add_frame_flags.

It is highly recommended to use the flag AV_BUFFERSRC_FLAG_KEEP_REF if you want to retain ownership of your AVFrame reference, or AV_BUFFERSRC_FLAG_PUSH to force immediate processing.

// Allocate and populate your frame
AVFrame *frame = av_frame_alloc();
// ... Fill frame->data and set frame->pts ...

// Push the frame into the filtergraph
int ret = av_buffersrc_add_frame_flags(buffersrc_ctx, frame, AV_BUFFERSRC_FLAG_KEEP_REF);
if (ret < 0) {
    // Error feeding the filtergraph
}

// If you did not use AV_BUFFERSRC_FLAG_KEEP_REF, the filtergraph takes ownership, 
// and the frame is automatically unreferenced. 
// If you did use the flag, you must free/unref your local reference manually when done:
av_frame_unref(frame);

4. Flushing the Filtergraph

When you reach the End of Stream (EOS) and have no more frames to send, you must notify the filtergraph by sending a NULL frame pointer. This informs downstream filters (like tempo or rate converters) to flush any remaining cached frames.

// Flush the filtergraph
int ret = av_buffersrc_add_frame_flags(buffersrc_ctx, NULL, 0);
if (ret < 0) {
    // Handle flush error
}