FFmpeg Realtime Filter: Match Real-Time Playback

By default, FFmpeg processes media files as fast as your CPU and GPU will allow, which can cause issues when testing live streams or generating simulated real-time outputs. The realtime video filter and arealtime audio filter throttle FFmpeg's internal processing pipeline to match the wall-clock playback speed of the media. This article explains how the filter works, how to use it in your commands, and how it compares to the common -re input flag.

Why Use the realtime Filter?

FFmpeg usually encodes offline media at speeds much faster than 1x. While ideal for standard file conversion, this can overload downstream network buffers or RTMP/SRT servers when simulating a live camera feed from a file. The realtime filter evaluates the presentation timestamps (PTS) of passing frames and pauses FFmpeg processing if the frames arrive faster than the actual clock time.

Basic Usage for Video

To slow down video processing to 1x real-time speed, insert realtime into your video filter chain using -vf:

ffmpeg -i input.mp4 -vf realtime -c:v libx264 -c:a aac -f flv rtmp://localhost/live/stream

In this command:

  • -i input.mp4: The source file being read.
  • -vf realtime: The filter checks the PTS of each frame and introduces delays to ensure 1 second of video takes at least 1 second of wall-clock time to process.

Basic Usage for Audio

If you are only processing an audio stream, use the audio equivalent, arealtime:

ffmpeg -i input.mp3 -af arealtime -c:a aac output.aac

Applying Real-Time Throttling to Both Streams

When processing both audio and video, you can throttle either stream, but throttling the video stream is generally sufficient. If you want strict enforcement on both, apply realtime to the video and arealtime to the audio simultaneously:

ffmpeg -i input.mp4 -vf realtime -af arealtime -c:v libx264 -c:a aac output.mp4

Alternatively, use filter_complex:

ffmpeg -i input.mp4 -filter_complex "[0:v]realtime[v];[0:a]arealtime[a]" -map "[v]" -map "[a]" -c:v libx264 -c:a aac output.mp4

Filter Options and Parameters

The realtime filter supports optional parameters to fine-tune behavior:

  • limit: Specifies the maximum acceptable pause time (in seconds). If the time delta is greater than this limit, the filter assumes a timestamp discontinuity and will not pause for the full duration. The default is 2 seconds.
  • speed: Controls the playback speed multiplier. The default is 1.0 (standard real-time). Setting this to 2.0 will process the stream at twice real-time speed.

Example using options:

ffmpeg -i input.mp4 -vf "realtime=limit=5:speed=1.0" -f null -

realtime Filter vs. the -re Option

While both restrict speed, they operate at different stages of the FFmpeg pipeline:

  • -re (Input Option): Placed before the input (ffmpeg -re -i input.mp4), it tells the demuxer to read packets from the disk at native frame rate. However, it can sometimes be imprecise with complex filter graphs or stream generators.
  • realtime (Filter Option): Operates inside the filter graph. This makes it suitable for synthetic inputs (such as testsrc or gradients) that do not read from a disk-based container, or inside mid-pipeline processing where filtering introduces variable latency.