FFmpeg VideoToolbox Encoder Settings and Parameters

This article provides a comprehensive overview of the configuration parameters available for Apple's VideoToolbox hardware encoder within FFmpeg. It covers essential settings for controlling bitrate, quality, profiles, and hardware acceleration options, enabling you to optimize video encoding performance on macOS and iOS devices.

Apple’s VideoToolbox framework allows FFmpeg to access hardware-accelerated video encoding for codecs like H.264 (h264_videotoolbox), HEVC (hevc_videotoolbox), and ProRes (prores_videotoolbox). Configuring these encoders properly requires using a mix of standard FFmpeg options and VideoToolbox-specific private flags.

Standard FFmpeg Parameters

These standard options are supported by the VideoToolbox wrappers and control the primary aspects of the encoding process:

-b:v <bitrate>: Sets the target average video bitrate (e.g., -b:v 5M for 5 Mbps).
-maxrate <bitrate>: Sets the maximum peak bitrate when using variable bitrate (VBR) control.
-bufsize <size>: Sets the receiver/decoder buffer size, which controls the rate control tightness.
-q:v <quality> or -global_quality <integer>: Sets the video quality scale. VideoToolbox uses this for variable bitrate quality-based encoding.
-g <integer>: Sets the Group of Pictures (GOP) size, which is the maximum distance between keyframes (I-frames).
-bf <integer>: Sets the number of B-frames. Note that older Mac hardware or specific profiles may restrict B-frame usage.

VideoToolbox Private Options

Private options are specific to the VideoToolbox encoder wrappers. You can apply these parameters using the -option value syntax in your FFmpeg command.

Parameter	Type	Description
`profile`	string	Sets the encoder profile (e.g., `baseline`, `main`, `high` for H.264; `main`, `main10` for HEVC).
`level`	string	Sets the encoder level (e.g., `4.1`, `5.1`, `auto`).
`allow_sw`	boolean	Allows fallback to software encoding if hardware acceleration is unavailable (`0` or `1`). Default is `0` (hardware only).
`require_hardware`	boolean	Forces the encoder to fail if hardware-accelerated encoding is not available (`0` or `1`). Default is `0`.
`realtime`	boolean	Optimizes the encoder for real-time latency, which is ideal for live streaming (`0` or `1`). Default is `0`.
`prioritize_speed`	boolean	Prioritizes encoding speed over visual quality (`0` or `1`). Default is `0`.
`max_ref_frames`	integer	Sets the maximum number of reference frames (H.264 only).
`frames_before`	integer	Specifies how many frames must be queued before the encoder starts outputting packets.

Color and Metadata Parameters

These parameters ensure that high dynamic range (HDR) or specific color spaces are passed correctly to the hardware encoder:

color_primaries: Sets color primaries (e.g., bt709, bt2020).
color_trc: Sets transfer characteristics (e.g., smpte2084 for HDR10, arib-std-b67 for HLG).
colorspace: Sets the matrix coefficients (e.g., bt2020nc, bt709).

Example Configuration Commands

Standard H.264 VBR Encoding:

ffmpeg -i input.mp4 -c:v h264_videotoolbox -b:v 4M -profile:v high -level 4.1 output.mp4

High-Quality HEVC (H.265) 10-bit Encoding:

ffmpeg -i input.mp4 -c:v hevc_videotoolbox -profile:v main10 -q:v 65 output.mp4

Low-Latency Real-Time Streaming:

ffmpeg -i input.mp4 -c:v h264_videotoolbox -b:v 2M -realtime 1 -allow_sw 1 output.mp4