Using avcodec_send_packet and avcodec_receive_frame

This article explains how to decode compressed multimedia streams using FFmpeg's modern decoding API. By utilizing avcodec_send_packet and avcodec_receive_frame, developers can implement a decoupled, push-and-pull decoding loop that correctly handles variable frame outputs, internal buffering, and end-of-stream flushing.

The Push-Pull Decoding Model

FFmpeg separates the input of compressed packets from the output of raw frames because the relationship between packets and frames is rarely one-to-one. A single video packet containing a B-frame might require additional packets before it can be decoded, or a single audio packet might produce multiple decoded audio frames.

To handle this, you "push" an AVPacket into the decoder context using avcodec_send_packet(), and then "pull" available AVFrame instances using avcodec_receive_frame() until the decoder requires more input.

Step-by-Step Implementation

The decoding loop consists of reading packets from an input format context, sending each packet to the codec context, and reading back all available frames.

#include <libavcodec/avcodec.h>

int decode_packet(AVCodecContext *codec_ctx, AVPacket *pkt, AVFrame *frame) {
    // 1. Send the packet to the decoder
    int ret = avcodec_send_packet(codec_ctx, pkt);
    if (ret < 0) {
        // Failed to send packet
        return ret;
    }

    // 2. Retrieve all available frames from the decoder
    while (ret >= 0) {
        ret = avcodec_receive_frame(codec_ctx, frame);
        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
            // EAGAIN: Decoder needs more packets to produce a frame
            // AVERROR_EOF: The decoder has been fully flushed
            return 0;
        } else if (ret < 0) {
            // An actual decoding error occurred
            return ret;
        }

        // Process the decoded frame (e.g., render, save, or convert)
        // Note: Remember to unreference the frame buffer after using it
        av_frame_unref(frame);
    }

    return 0;
}

Understanding Return Values

Both functions communicate the decoder's state through specific integer return codes:

• 0 (Success): The operation succeeded. For avcodec_send_packet, the packet was accepted. For avcodec_receive_frame, a valid frame was populated.
• AVERROR(EAGAIN):
  • In avcodec_send_packet: The decoder's internal buffer is full. You must drain frames using avcodec_receive_frame before sending new packets.
  • In avcodec_receive_frame: The decoder has exhausted its current output and requires new input packets before it can output another frame.
• AVERROR_EOF: The decoder has been flushed, and no more frames will be produced.
• Negative Values (Other errors): Indicates a fatal decoding or memory error.

Flushing the Decoder

When you reach the end of the input stream (av_read_frame returns AVERROR_EOF), delayed frames may still be cached inside the decoder. You must flush the decoder to retrieve them:

1. Call avcodec_send_packet(codec_ctx, NULL) to enter draining mode.
2. Run your avcodec_receive_frame() loop repeatedly until it returns AVERROR_EOF.
3. If you intend to reuse the codec context for seeking, call avcodec_flush_buffers(codec_ctx) to reset internal states.