Extract Frames with FFmpeg Buffersink in C

This guide explains how to extract processed audio and video frames from an FFmpeg filtergraph using buffersink or abuffersink in C. You will learn the core workflow of feeding decoded data into an input filter, polling the sink filter using av_buffersink_get_frame(), properly handling loop control codes like EAGAIN and AVERROR_EOF, and managing frame memory.

The Filtergraph Data Flow

Extracting filtered frames relies on a push-pull mechanism between two special endpoints in libavfilter:

• Source (buffer / abuffer): Feeds raw decoded frames into the graph via av_buffersrc_add_frame_flags().
• Sink (buffersink / abuffersink): Delivers the transformed frames to your application via av_buffersink_get_frame().

Filtering is a pull-driven process: calling the sink triggers the necessary processing steps backward through the graph up to the source.

Step-by-Step Implementation

1. Allocate an Output Frame

Create an AVFrame pointer that will store the filtered data returned by the sink:

AVFrame *filt_frame = av_frame_alloc();
if (!filt_frame) {
    // Handle memory allocation failure
}

2. Push an Unfiltered Frame to the Source

Before pulling filtered data, supply an input frame (frame) to your source filter context (buffersrc_ctx):

int ret = av_buffersrc_add_frame_flags(buffersrc_ctx, frame, AV_BUFFERSRC_FLAG_KEEP_REF);
if (ret < 0) {
    // Handle error pushing frame to filtergraph
}

Note: To flush the filtergraph at the end of a stream, pass NULL instead of frame.

3. Pull Filtered Frames from the Sink

A single input frame can produce zero, one, or multiple output frames depending on the configured filters (such as fps, decimate, or aresample). Therefore, read from the sink context (buffersink_ctx) inside a while loop:

while (1) {
    ret = av_buffersink_get_frame(buffersink_ctx, filt_frame);
    if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
        // EAGAIN: Need to feed more frames to the source to get an output
        // AVERROR_EOF: The graph is fully drained after a flush
        break;
    }
    if (ret < 0) {
        // Fatal error occurred during filtering
        break;
    }

    // Process the filtered frame (e.g., render, write to disk, encode)
    process_frame(filt_frame);

    // Release the frame's buffer reference before the next iteration
    av_frame_unref(filt_frame);
}

Audio vs. Video Buffersinks

The retrieval logic remains identical for both audio and video, with the following differences in configuration:

If an audio encoder requires an exact number of samples per frame (e.g., AAC requires 1024 samples), call av_buffersink_set_frame_size(buffersink_ctx, frame_size) after configuring the graph.

Clean Up

Once processing is complete, release the temporary frame:

av_frame_free(&filt_frame);