FFmpeg Scene Change Detection: Extracting Frames

Extracting specific frames from a video based on scene transitions is a powerful capability of FFmpeg. This guide explains how to use FFmpeg’s select filter alongside the scene detection parameter to identify major visual changes and export those key moments as individual image files. You will learn the exact command syntax, how the detection threshold works, and how to customize the sensitivity for your videos.

To extract frames at scene changes, you use the select video filter with the scene evaluation variable. Here is the standard command:

ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)'" -fps_mode vfr output_%03d.png

How the Command Works

• -i input.mp4: Specifies your input video file.
• -vf "select='gt(scene,0.4)'": This is the core filter. The select filter decides which frames to keep. The expression gt(scene,0.4) stands for “greater than.” It calculates a value between 0 and 1 representing the difference between the current frame and the previous one. If the difference is greater than 0.4 (representing a 40% change), FFmpeg selects the frame.
• -fps_mode vfr: Sets the output frame rate to variable frame rate (this replaces the legacy -vsync vfr option). This is crucial because it prevents FFmpeg from duplicating the selected frames to match the original video’s constant frame rate.
• output_%03d.png: The naming pattern for the exported images, which will be numbered sequentially (e.g., output_001.png, output_002.png).

Adjusting Sensitivity

The decimal value in gt(scene,0.4) controls how sensitive the detection is:

• High Sensitivity (0.1 to 0.3): Useful for detecting subtle transitions, fast camera movements, or soft cross-fades. This will generate a large number of output images.
• Medium Sensitivity (0.4 to 0.5): The standard range for hard cuts and distinct scene changes in most movies and presentations.
• Low Sensitivity (0.6 to 0.7): Only detects massive visual changes, such as cutting from a very dark scene to a bright white screen.

Including Timestamps in the Console

If you want to view the exact timestamps of when the scene changes occurred while extracting the frames, add the showinfo filter to your command:

ffmpeg -i input.mp4 -vf "select='gt(scene,0.4)',showinfo" -fps_mode vfr output_%03d.png

This will output detailed frame metadata, including presentation timestamps (PTS), directly to your terminal.