FFmpeg Memory Management: av_packet_free and av_frame_free

Proper memory management in FFmpeg's C API requires understanding reference counting and the distinction between freeing internal buffers versus freeing the container structs themselves. This guide covers the correct lifecycle of AVPacket and AVFrame, explaining how to use av_packet_free, av_frame_free, and their _unref counterparts to prevent both memory leaks and segmentation faults in video processing pipelines.

The Dual-Layer Architecture of Frames and Packets

Both AVPacket and AVFrame consist of two parts:

  1. The Container Struct: The AVPacket or AVFrame structure containing metadata (e.g., PTS, DTS, width, height, format).
  2. The Underlying Buffer: The actual payload (compressed bitstream data for packets, raw planar/packed audio-video data for frames), which is managed via AVBufferRef reference counting.

Failing to distinguish between clearing the data payload and deallocating the structure itself is the leading cause of memory leaks in FFmpeg applications.

Freeing vs. Unreferencing

Loop Processing: av_frame_unref and av_packet_unref

Inside decoding, encoding, or read loops, do not destroy the container structs repeatedly. Instead, wipe the payload and reset the reference counts:

  • av_packet_unref(AVPacket *pkt): Decrements the internal buffer reference counter and zeroes out the fields in pkt.
  • av_frame_unref(AVFrame *frame): Decrements the underlying buffer references, resets frame metadata, and clears plane pointers.

Using these functions allows you to reuse the allocated struct instance across multiple iterations without leaking the data buffers associated with each processed frame or packet.

Pipeline Teardown: av_frame_free and av_packet_free

When the processing pipeline completes or encounters a fatal error, the container pointers must be released.

  • av_packet_free(AVPacket **pkt)
  • av_frame_free(AVFrame **frame)

Both functions take a double pointer (&ptr). They automatically call av_*_unref to release any remaining payload references, free the container struct memory from the heap, and set the passed pointer to NULL to prevent dangling references and double-free vulnerabilities.

The safest approach is to allocate structures once before your processing loop, clear them within the loop, and completely free them in the cleanup phase.

#include <libavcodec/avcodec.h>

void process_media(AVFormatContext *fmt_ctx, AVCodecContext *codec_ctx) {
    // 1. Allocate containers once
    AVPacket *pkt = av_packet_alloc();
    AVFrame *frame = av_frame_alloc();

    if (!pkt || !frame) {
        // Allocation failure handling
        av_packet_free(&pkt);
        av_frame_free(&frame);
        return;
    }

    // 2. Processing loop
    while (av_read_frame(fmt_ctx, pkt) >= 0) {
        if (pkt->stream_index == 0) {
            int ret = avcodec_send_packet(codec_ctx, pkt);
            if (ret >= 0) {
                while (avcodec_receive_frame(codec_ctx, frame) >= 0) {
                    // Process frame data here...

                    // Clear frame data for next receive call
                    av_frame_unref(frame);
                }
            }
        }
        // Clear packet data for next read call
        av_packet_unref(pkt);
    }

    // 3. Final teardown: unreferences remaining buffers and frees structs
    av_packet_free(&pkt);
    av_frame_free(&frame);
}

Common Mistakes to Avoid

  1. Allocating on the Stack: In modern FFmpeg (libavutil > 56, libavcodec > 58), sizeof(AVPacket) and sizeof(AVFrame) are not part of the public ABI. Declaring AVPacket pkt; on the stack is deprecated and can lead to memory corruption. Always allocate using av_packet_alloc() and av_frame_alloc().
  2. Calling free() or av_free() Directly: Never pass an AVFrame* or AVPacket* directly to av_free() or free(). Doing so frees the struct wrapper without decrementing reference counts on internal data buffers, resulting in severe memory leaks.
  3. Reallocating Inside Loops: Calling av_packet_alloc() or av_frame_alloc() inside a loop without matching _free calls will rapidly exhaust system memory. Allocate once outside the loop and use _unref inside.
  4. Ignoring Return Values: FFmpeg functions like avcodec_receive_frame transfer buffer ownership. If the function returns an error (such as AVERROR(EAGAIN)), ensure you still appropriately unreference buffers if data was partially written or if your architecture reuses custom buffers.