How to Use avcodec_send_packet and avcodec_receive_frame

This guide explains how to decode compressed media packets into raw audio or video frames using the modern FFmpeg API. It covers the core decoding loop utilizing avcodec_send_packet and avcodec_receive_frame, detail how to handle specific API return codes, and explains how to properly flush the decoder at the end of a stream.

The Decoding Workflow

FFmpeg uses a push/pull model for decoding. You send compressed data (an AVPacket) to the codec context using avcodec_send_packet, and then retrieve decoded raw data (an AVFrame) using avcodec_receive_frame.

Because of frame reordering (like B-frames in video) or compression ratios, the relationship between input packets and output frames is not 1:1. Sending one packet may yield zero, one, or multiple frames.

Step-by-Step Decoding Loop

To decode successfully, you must run a nested loop. Send your packet to the decoder first, and then continuously attempt to pull frames until the decoder requires more input.

int decode_packet(AVCodecContext *dec_ctx, AVPacket *pkt, AVFrame *frame) {
    int response = avcodec_send_packet(dec_ctx, pkt);
    
    if (response < 0) {
        // Error sending the packet to the decoder
        return response;
    }

    while (response >= 0) {
        response = avcodec_receive_frame(dec_ctx, frame);
        if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
            // EAGAIN: decoder needs more packets to output a frame
            // AVERROR_EOF: the decoder has been fully flushed
            break;
        } else if (response < 0) {
            // Legitimate decoding error
            return response;
        }

        // At this point, you have a successfully decoded frame
        // Process your frame here (e.g., render it or write to file)
        
        av_frame_unref(frame); // Clear frame properties before reuse
    }
    return 0;
}

Handling Return Values

Understanding the specific return values of both functions is critical for preventing memory leaks and application crashes:

• avcodec_send_packet return values:
  • 0: Success. The packet was successfully accepted by the decoder.
  • AVERROR(EAGAIN): The decoder’s internal buffer is full. You must read outstanding frames using avcodec_receive_frame before you can send new packets.
  • AVERROR_EOF: The decoder has been flushed, and no new packets can be sent.
  • AVERROR(EINVAL): Codec not opened, or it is configured for encoding, not decoding.
• avcodec_receive_frame return values:
  • 0: Success. A frame was returned.
  • AVERROR(EAGAIN): No frame is currently available. You must send more packets via avcodec_send_packet to obtain more output.
  • AVERROR_EOF: The decoder has been fully flushed and will output no more frames.
  • AVERROR(EINVAL): Codec not opened, or it is configured for encoding.

Flushing the Decoder

At the end of your video or audio stream, some decoded frames may still be buffered inside the decoder’s pipeline. To retrieve these remaining “delayed” frames, you must flush the decoder:

1. Pass NULL (or an empty AVPacket with its data and size fields set to 0/NULL) to avcodec_send_packet.
2. Enter the avcodec_receive_frame loop.
3. Continue calling avcodec_receive_frame until it returns AVERROR_EOF. This signals that all buffered frames have been extracted.