Set FFmpeg Named Pipe Buffer Size to Prevent Data Loss

Using named pipes (FIFOs) with FFmpeg is an efficient way to stream real-time video and audio data between applications without writing to physical storage. However, because named pipes rely on limited system memory buffers, a mismatch in speed between the writer and reader can easily lead to overflow, underflow, and severe data loss. This article explains how to configure buffer sizes both within FFmpeg and at the operating system level to ensure reliable, high-performance data transmission.

Method 1: Increase the Thread Queue Size (Input Pipes)

When FFmpeg reads from a named pipe, it may temporarily fall behind during heavy encoding tasks, causing the pipe’s OS buffer to overflow. You can prevent this by increasing the demuxer’s thread queue size. This forces FFmpeg to allocate a larger in-memory queue to buffer incoming packets.

Add the -thread_queue_size option immediately before the input file option:

ffmpeg -thread_queue_size 1024 -i /path/to/named_pipe -c:v libx264 output.mp4

• Default Value: Usually 8 or 16 packets.
• Recommended Value: Try 512, 1024, or 2048 depending on your system’s RAM and the bitrate of your stream.

Method 2: Use the FIFO Muxer (Output Pipes)

If FFmpeg is writing to a named pipe and the receiving application is slow to read the data, FFmpeg will block or drop frames. To prevent this, use FFmpeg’s native fifo muxer. This runs the output writing process in a separate thread with its own configurable queue.

ffmpeg -i input.mp4 -f fifo -fifo_format mpegts -map 0 -queue_size 2000 /path/to/named_pipe

• -f fifo: Enables the FIFO muxer.
• -fifo_format mpegts: Specifies the actual format of the output stream (replace mpegts with your desired format, like flv or nut).
• -queue_size 2000: Sets the maximum number of packets to buffer in memory before dropping data.

Method 3: Adjust the OS Pipe Buffer Size (Linux)

By default, the Linux kernel limits a named pipe’s capacity to 64 KB. For high-definition video, this buffer is filled almost instantly. You can bypass this kernel limitation by routing your data through mbuffer or pv (Pipe Viewer), which can allocate massive RAM buffers.

Using mbuffer

mbuffer acts as an intermediary buffer that can hold megabytes of data in system memory.

1. Create your named pipe:

   mkfifo /tmp/my_pipe

2. Run FFmpeg reading from standard input, fed by mbuffer reading from the pipe:

   mbuffer -m 64M -I /tmp/my_pipe | ffmpeg -i - -c:v copy output.mp4

   (In this example, -m 64M sets a 64 Megabyte buffer in RAM).

Using pv

If mbuffer is not installed, you can use pv with a custom buffer size:

pv -B 32M /tmp/my_pipe | ffmpeg -i - -c:v copy output.mp4

(Here, -B 32M allocates a 32 Megabyte buffer).