Fix FFmpeg Timestamps with setpts After Selecting Frames

When you select specific frames in FFmpeg using filters like select, the original presentation timestamps (PTS) of those frames are preserved. This leaves gaps in the timeline where the dropped frames used to be, resulting in frozen video, stuttering, or playback errors. To fix this, you must use the setpts filter to recalculate and normalize the timestamps. This article explains how to use the setpts filter to create a continuous, smooth timeline after frame selection.

The Problem: Why Timestamps Break

If you filter a video to keep only every 10th frame, FFmpeg keeps the timestamps of those frames (e.g., 0s, 0.4s, 0.8s) but drops the frames in between. The player expects a continuous stream, so it will pause on the first frame for 0.4 seconds, causing severe lag.

To solve this, you must rewrite the timestamps so that the selected frames play sequentially without gaps.

The Solution: Using setpts=N/FRAME_RATE/TB

The standard expression to correct timestamps after selecting frames is:

setpts=N/FRAME_RATE/TB

Here is what these variables mean: * N: The sequential count of the input frame, starting from 0. After filtering, N becomes the new index of the remaining frames. * FRAME_RATE: The frame rate of the video stream. * TB: The Time Base of the stream.

By dividing the frame index by the frame rate and the time base, FFmpeg generates a perfectly continuous, gapless sequence of timestamps based on your output frame rate.

Practical Examples

Example 1: Selecting Every 5th Frame

To extract every 5th frame and pack them together with continuous timestamps:

ffmpeg -i input.mp4 -vf "select='not(mod(n\,5))',setpts=N/FRAME_RATE/TB" output.mp4

Example 2: Extracting Only Keyframes (I-Frames)

If you want to extract and play only the keyframes in a video, the gap between frames is highly variable. You must reset the PTS to make the keyframes play smoothly:

ffmpeg -i input.mp4 -vf "select='eq(pict_type\,I)',setpts=N/FRAME_RATE/TB" output.mp4

Example 3: Handling Both Video and Audio

If you select frames and need to keep the audio synced (or similarly decimated), you must apply the audio equivalent filter, asetpts, using the sample rate (SR):

ffmpeg -i input.mp4 -filter_complex "[0:v]select='not(mod(n\,2))',setpts=N/FRAME_RATE/TB[v];[0:a]aselect='not(mod(n\,2))',asetpts=N/SR/TB[a]" -map "[v]" -map "[a]" output.mp4

Alternative: Using PTS-STARTPTS (For Trimming Only)

If you are not dropping frames from the middle of the video but are instead cutting/trimming a chunk from the beginning, you should use:

setpts=PTS-STARTPTS

This simply shifts the timestamps of the video so that the first output frame starts exactly at 0, preserving the original relative timing of all subsequent frames. Do not use this method if you have dropped individual frames throughout the video, as the gaps will remain.