FFmpeg Hardware Decoding to Shared Memory

This article explains how to configure and run an FFmpeg command that leverages hardware acceleration to decode a video file and stream the decoded raw video frames directly into a shared memory segment. This approach minimizes CPU usage and eliminates disk I/O bottlenecks, making it ideal for high-performance video processing pipelines and inter-process communication (IPC) on Linux systems using /dev/shm.

Step 1: Create a Named Pipe in Shared Memory

The most efficient way to stream frames from FFmpeg to another application via shared memory without writing custom C bindings is to use a named pipe (FIFO) inside the Linux RAM-backed shared memory directory (/dev/shm).

Run the following command in your terminal to create the named pipe:

mkfifo /dev/shm/video_pipe

Step 2: Run the FFmpeg Command

Depending on your graphics hardware, choose the appropriate command below to decode the video using hardware acceleration and output the raw frames to the shared memory pipe.

Option A: For NVIDIA GPUs (CUDA/NVDEC)

This command uses NVIDIA’s hardware decoder to process the video, transfers the decoded frames from GPU memory (VRAM) back to system memory (RAM), and writes them to the shared memory pipe.

ffmpeg -y \
  -hwaccel cuda \
  -hwaccel_output_format cuda \
  -i input.mp4 \
  -vf "hwdownload,format=nv12" \
  -f rawvideo \
  -pix_fmt nv12 \
  /dev/shm/video_pipe

Option B: For Intel/AMD GPUs (VAAPI)

If you are using Intel integrated graphics or AMD GPUs on Linux, use the VAAPI hardware accelerator instead:

ffmpeg -y \
  -hwaccel vaapi \
  -hwaccel_device /dev/dri/renderD128 \
  -hwaccel_output_format vaapi \
  -i input.mp4 \
  -vf "hwdownload,format=nv12" \
  -f rawvideo \
  -pix_fmt nv12 \
  /dev/shm/video_pipe

Parameter Breakdown

• -hwaccel cuda / -hwaccel vaapi: Activates the respective hardware acceleration API to decode the video stream on the GPU.
• -hwaccel_output_format cuda/vaapi: Keeps the decoded frames in GPU memory for initial processing.
• -i input.mp4: Specifies the input video file.
• -vf "hwdownload,format=nv12": This filter chain is critical. It downloads the decoded frame from GPU VRAM to system RAM, then converts it to the standard nv12 pixel format.
• -f rawvideo: Tells FFmpeg to output containerless, uncompressed raw video frames.
• -pix_fmt nv12: Forces the output pixel format to NV12 (YUV 4:2:0), which matches the hardware decoder’s native output.
• /dev/shm/video_pipe: The destination path pointing to our named pipe inside the shared memory space.

Step 3: Consuming the Shared Memory Stream

Because /dev/shm/video_pipe is a FIFO pipe in RAM, FFmpeg will block (pause) after writing the first frame until a consumer application starts reading from the pipe.

You can read the raw frames from /dev/shm/video_pipe using any programming language (such as Python, C++, or Rust) by opening the file path and reading the exact bytes per frame. For example, for an NV12-formatted frame, the frame size in bytes is calculated as:

\[\text{Frame Size} = \text{Width} \times \text{Height} \times 1.5\]

Once your consumer application reads the bytes, they are immediately available in system memory for processing, machine learning inference, or rendering.