Feed in-memory buffer to FFmpeg libavformat

This article explains how to feed media data from an in-memory buffer directly into FFmpeg's libavformat using a custom I/O read callback. You will learn how to configure an AVIOContext with your own read function and associate it with an AVFormatContext to demux and decode media data without using physical file paths.

The Custom I/O Mechanism

By default, FFmpeg's avformat_open_input expects a file path or network URL to open a media stream. However, if your media data is already loaded in memory (such as a buffer received over a custom network protocol, encrypted storage, or an IPC pipe), you must intercept FFmpeg's read requests.

This is achieved by creating a custom AVIOContext using avio_alloc_context and assigning it to the pb field of your AVFormatContext.

Step-by-Step Implementation

To feed in-memory data to libavformat, you must define a custom state structure, implement a read callback, allocate a buffer for FFmpeg's internal use, and initialize the contexts.

1. Define the Buffer State Structure

You need a custom structure to keep track of the pointer to your in-memory buffer and the amount of data remaining.

#include <libavformat/avformat.h>
#include <libavutil/mem.h>
#include <string.h>

struct BufferData {
    uint8_t *ptr;
    size_t size; // Number of bytes remaining in the buffer
};

2. Implement the Read Callback

The read callback is called by FFmpeg whenever it needs more data to parse the container or extract packets. It must copy data from your custom buffer to FFmpeg's internal buffer.

int read_packet(void *opaque, uint8_t *buf, int buf_size) {
    struct BufferData *bd = (struct BufferData *)opaque;
    
    // Determine how much data can actually be read
    buf_size = FFMIN(buf_size, bd->size);

    if (buf_size <= 0) {
        return AVERROR_EOF; // Return End of File
    }

    // Copy data to FFmpeg's internal buffer
    memcpy(buf, bd->ptr, buf_size);
    bd->ptr  += buf_size;
    bd->size -= buf_size;

    return buf_size;
}

3. Allocate and Bind the Contexts

Configure the AVFormatContext to use your custom read callback instead of its default file I/O operations.

int open_memory_buffer(uint8_t *my_buffer, size_t my_buffer_len) {
    AVFormatContext *fmt_ctx = avformat_alloc_context();
    if (!fmt_ctx) {
        return AVERROR(ENOMEM);
    }

    // Initialize your custom buffer state struct
    struct BufferData bd = {
        .ptr = my_buffer,
        .size = my_buffer_len
    };

    // Allocate the buffer that AVIOContext will use internally
    size_t avio_ctx_buffer_size = 4096;
    uint8_t *avio_ctx_buffer = av_malloc(avio_ctx_buffer_size);
    if (!avio_ctx_buffer) {
        avformat_free_context(fmt_ctx);
        return AVERROR(ENOMEM);
    }

    // Allocate the AVIOContext
    AVIOContext *avio_ctx = avio_alloc_context(
        avio_ctx_buffer, 
        avio_ctx_buffer_size,
        0,                  // Write flag (0 = read-only, 1 = write-only)
        &bd,                // Opaque pointer passed to callbacks
        &read_packet,       // Read callback
        NULL,               // Write callback (NULL for read-only)
        NULL                // Seek callback (optional, NULL if seeking is not supported)
    );

    if (!avio_ctx) {
        av_free(avio_ctx_buffer);
        avformat_free_context(fmt_ctx);
        return AVERROR(ENOMEM);
    }

    // Bind the custom I/O context to the format context
    fmt_ctx->pb = avio_ctx;

    // Open the input. The URL parameter must be NULL or a dummy string.
    int ret = avformat_open_input(&fmt_ctx, NULL, NULL, NULL);
    if (ret < 0) {
        // Cleaning up avio_ctx and fmt_ctx is required on failure
        avio_context_free(&avio_ctx);
        avformat_close_input(&fmt_ctx);
        return ret;
    }

    // Now you can call avformat_find_stream_info and read frames
    ret = avformat_find_stream_info(fmt_ctx, NULL);
    if (ret < 0) {
        avformat_close_input(&fmt_ctx);
        avio_context_free(&avio_ctx);
        return ret;
    }

    // Process your media here...

    // Cleanup resources
    avformat_close_input(&fmt_ctx);
    
    // Note: av_free(avio_ctx_buffer) is handled automatically inside avio_context_free
    avio_context_free(&avio_ctx); 

    return 0;
}

Crucial Considerations

Buffer Ownership: The pointer passed to avio_alloc_context (avio_ctx_buffer) must be allocated via av_malloc. FFmpeg takes ownership of this buffer and will free it automatically when you call avio_context_free. Do not manually free it while the context is active.
Probing Formats: Because you are passing a NULL URL to avformat_open_input, FFmpeg cannot guess the container format from a file extension. It will rely purely on probing the first bytes returned by your read callback. If your format is headerless (e.g., raw AAC, PCM, or H.264 streams), you may need to manually specify the input format using the AVInputFormat parameter in avformat_open_input.
Seeking: If your in-memory buffer allows seeking, you can implement a seek callback and pass it as the last argument to avio_alloc_context. If you do not provide a seek callback, FFmpeg will only read sequentially.