How to Build Video Filtergraphs with ffmpeg-python

This article provides a practical guide on constructing simple and complex video filtergraphs using the ffmpeg-python library. You will learn the core concepts of chaining operations, handling multiple video and audio streams, applying common transformations like scaling and cropping, and combining streams using complex filters such as overlays.

Understanding the Basics

The ffmpeg-python library provides Python bindings for FFmpeg, allowing you to build complex command-line arguments using a fluent, node-based API. In FFmpeg, a filtergraph is a directed graph of connected filters. In ffmpeg-python, each filter function takes an incoming stream node as an input and returns a new stream node.

Before starting, ensure that both the FFmpeg binary is installed on your system and the Python package is installed:

pip install ffmpeg-python

Creating a Simple Linear Filtergraph

A linear filtergraph applies a sequence of filters to a single input stream, passing the result from one filter directly into the next.

import ffmpeg

# Define the input
input_stream = ffmpeg.input('input.mp4')

# Build a linear filter chain: trim -> scale -> set presentation timestamp
video = (
    input_stream.video
    .trim(start=5, end=15)
    .setpts('PTS-STARTPTS')
    .filter('scale', 1280, 720)
    .filter('fps', fps=30)
)

# Extract and trim the corresponding audio
audio = (
    input_stream.audio
    .filter('atrim', start=5, end=15)
    .filter('asetpts', 'PTS-STARTPTS')
)

# Combine video and audio into an output node and run
output = ffmpeg.output(video, audio, 'output_linear.mp4')
ffmpeg.run(output)

Building Complex Filtergraphs with Multiple Inputs

Complex filtergraphs involve multiple inputs, outputs, or branching paths. A common use case is creating a picture-in-picture (PiP) effect using the overlay filter.

import ffmpeg

# Load primary and overlay inputs
main_video = ffmpeg.input('main_background.mp4')
pip_video = ffmpeg.input('secondary_camera.mp4')

# Scale down the overlay stream
pip_scaled = pip_video.filter('scale', 320, 240)

# Apply the overlay filter: place secondary stream at coordinates (x=50, y=50)
composed_video = ffmpeg.filter(
    [main_video.video, pip_scaled],
    'overlay',
    x=50,
    y=50
)

# Render the output using the processed video and the main audio
output = ffmpeg.output(
    composed_video,
    main_video.audio,
    'output_pip.mp4',
    vcodec='libx264',
    acodec='aac'
)

ffmpeg.run(output)

Splitting and Branching Streams

You can use the .split() filter to route a single video stream into multiple parallel filter paths, for example, to create a side-by-side comparison or multiple outputs.

import ffmpeg

input_stream = ffmpeg.input('sample.mp4')

# Split the video stream into two identical branches
split = input_stream.video.split()
branch1 = split[0]
branch2 = split[1]

# Apply distinct filters to each branch
original_styled = branch1.filter('scale', 640, 480)
grayscale_styled = branch2.filter('scale', 640, 480).filter('hue', s=0)

# Concatenate or stack them horizontally using hstack
stacked = ffmpeg.filter([original_styled, grayscale_styled], 'hstack')

# Output the result
output = ffmpeg.output(stacked, input_stream.audio, 'side_by_side.mp4')
ffmpeg.run(output)

Inspecting the Generated Command

To debug a filtergraph without running the process, you can compile the node graph into an argument list:

args = ffmpeg.compile(output)
print(" ".join(args))

This prints the raw ffmpeg command, allowing you to inspect the -filter_complex arguments generated by ffmpeg-python to verify connection routing and parameters.