Use FFmpeg setpts to Fix Timestamps After Frame Selection

When you select or drop frames in FFmpeg using filters like select or aselect, the remaining frames keep their original Presentation Time Stamps (PTS). This creates gaps in the timeline, resulting in frozen frames, stuttering, or playback errors. This article demonstrates how to use the setpts filter to recalculate and reset these timestamps, ensuring a smooth, continuous output video.

Why Timestamps Break During Frame Selection

FFmpeg relies on Presentation Time Stamps (PTS) to know exactly when to display each frame. If you use a filter to drop every second frame, the timestamps of the remaining frames do not automatically shift forward. For example, if you keep frames at 0s, 2s, and 4s, the player will freeze for two seconds between each frame.

To fix this, you must chain the setpts filter immediately after your frame selection filter to rewrite the timestamps.

If you are dropping frames arbitrarily (such as keeping every 5th frame or dropping duplicate frames), the most reliable way to fix timestamps is to generate a completely new timeline based on the output frame index.

Use the following filter formula:

ffmpeg -i input.mp4 -vf "select='not(mod(n,5))',setpts=N/FRAME_RATE/TB" output.mp4

How it works:

  • select='not(mod(n,5))': Keeps only every 5th frame.
  • setpts=N/FRAME_RATE/TB:
    • N: The sequential index of the current frame (0, 1, 2, 3...).
    • FRAME_RATE: The target output frame rate.
    • TB: The stream's time base.
    • This mathematical formula calculates a perfectly spaced, gapless timeline for the remaining frames.

Method 2: Resetting the Start Time After Trimming

If you are selecting a continuous block of video from the middle of a file (for example, keeping only the video between the 10th and 20th seconds), the frames will still carry their original starting timestamps. This causes the output video to start with several seconds of black screen or a frozen frame.

To shift the selected clip so that it starts immediately at 0 seconds, use the STARTPTS variable:

ffmpeg -i input.mp4 -vf "select='between(t,10,20)',setpts=PTS-STARTPTS" output.mp4

How it works:

  • select='between(t,10,20)': Selects frames between second 10 and second 20.
  • setpts=PTS-STARTPTS: Subtracts the timestamp of the very first selected frame (STARTPTS) from all subsequent frames (PTS). This offsets the entire selected segment so the output video starts playing at 0:00.

Method 3: Adjusting Speed and Timestamps Together

If you are selecting frames to create a fast-motion or timelapse effect, you must scale the timestamps to match the new speed. If you select 1 out of every 10 frames, the video should play 10 times faster.

ffmpeg -i input.mp4 -vf "select='not(mod(n,10))',setpts=0.1*PTS" output.mp4

How it works:

  • setpts=0.1*PTS: Multiplies each presentation timestamp by 0.1. This compresses the timeline, making the remaining frames play back 10 times faster to fill the gaps left by the deleted frames.