How to Read Raw Video from Network Socket with FFmpeg

This article explains how to use FFmpeg to receive and process raw, uncompressed video streams transmitted over a network socket using TCP or UDP. Because raw video lacks header metadata like resolution, frame rate, or pixel format, you will learn how to explicitly define these parameters in your FFmpeg command so the incoming stream can be decoded and processed correctly.

The Challenge of Raw Video

When streaming containers like MP4 or MKV, the file headers tell the player the video's resolution, frame rate, and pixel format. Raw video (such as raw YUV or RGB) contains none of this information—it is just a continuous stream of pixel bytes.

To read this stream over a network socket, you must pass the format details to FFmpeg as input options before specifying the input socket URI.

FFmpeg Command Structure

The basic template for reading raw video from a socket is:

ffmpeg [input_options] -i [socket_protocol]://[address]:[port] [output_options] [output_destination]

Required Input Options

Because the input is raw, you must define these four options before -i:

  • -f rawvideo: Forces FFmpeg to demux the input as raw video.
  • -pix_fmt: Defines the pixel format of the incoming stream (e.g., yuv420p, rgb24, uyvy422).
  • -s (or -video_size): Specifies the resolution of the video (e.g., 1920x1080).
  • -r (or -framerate): Specifies the frame rate of the incoming stream (e.g., 30 or 60).

Practical Examples

1. Reading Raw Video via TCP (FFmpeg as a Listener)

If your video source is sending data to a specific port on your machine, you can configure FFmpeg to listen for the incoming connection. The ?listen=1 parameter tells FFmpeg to act as a TCP server.

ffmpeg -f rawvideo -pix_fmt yuv420p -s 1920x1080 -r 30 -i tcp://0.0.0.0:12345?listen=1 -c:v libx264 output.mp4
  • tcp://0.0.0.0:12345?listen=1: Listens on port 12345 on all local network interfaces.
  • -c:v libx264 output.mp4: Encodes the incoming raw stream to H.264 and saves it to an MP4 file.

2. Reading Raw Video via TCP (FFmpeg as a Client)

If the video source is already hosting a TCP server on another device (e.g., IP 192.168.1.50 on port 5000), FFmpeg can connect to it directly as a client.

ffmpeg -f rawvideo -pix_fmt rgb24 -s 1280x720 -r 60 -i tcp://192.168.1.50:5000 -c:v rawvideo -f sdl "Raw Video Playback"
  • tcp://192.168.1.50:5000: Connects to the remote server.
  • -f sdl "Raw Video Playback": Renders the incoming video stream directly to a local window for real-time viewing.

3. Reading Raw Video via UDP

UDP is ideal for low-latency streaming. Since UDP is connectionless, you do not need a listen parameter; FFmpeg will simply bind to the port and ingest any packets sent to it.

ffmpeg -f rawvideo -pix_fmt yuv420p -s 1920x1080 -r 25 -i udp://0.0.0.0:12345 -f null -
  • udp://0.0.0.0:12345: Binds to UDP port 12345.
  • -f null -: Benchmarks the stream by discarding the output, allowing you to test the incoming network bandwidth and decoding performance without writing to disk.

Troubleshooting Tips

  • Green or Distorted Video: If the output video is green, scrambled, or diagonally skewed, your -pix_fmt or -s (resolution) parameters do not match the sender's actual stream configuration.
  • Packet Loss on UDP: Raw video requires high bandwidth. If you experience dropped frames over UDP, increase the socket buffer size by appending the buffer_size option to your URI: udp://0.0.0.0:12345?buffer_size=10000000 (sets buffer to 10MB).