Compile FFmpeg with VideoToolbox on macOS

Compiling FFmpeg with Apple’s VideoToolbox framework allows you to leverage hardware-accelerated video encoding and decoding on macOS. This guide provides a straightforward, step-by-step walkthrough to install the necessary dependencies, configure the FFmpeg source code with the VideoToolbox flag, and compile the final binary on your Mac.

Step 1: Install Prerequisites

Before compiling FFmpeg, you need to install the Xcode Command Line Tools and the Homebrew package manager to download essential build dependencies.

Open your Terminal and install the Xcode Command Line Tools:
```
xcode-select --install
```

Install Homebrew (if you do not have it already):

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Install the required compilation tools and libraries via Homebrew:
```
brew install git pkg-config nasm yasm
```

Step 2: Clone the FFmpeg Source Code

Clone the official FFmpeg git repository to your local machine and navigate into the directory:

git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg

To use a specific stable release instead of the master branch, you can check out a release tag (e.g., n7.0):

git checkout n7.0

Step 3: Configure the Build with VideoToolbox

Run the ./configure script to prepare the build. The crucial flag here is --enable-videotoolbox, which enables hardware acceleration for macOS.

./configure \
  --prefix=/usr/local \
  --enable-gpl \
  --enable-nonfree \
  --enable-videotoolbox \
  --enable-audiotoolbox \
  --disable-shared \
  --enable-static

Configuration Flags Explained:

--prefix=/usr/local: Sets the installation directory.
--enable-videotoolbox: Enables Apple’s hardware-accelerated video encoding and decoding.
--enable-audiotoolbox: Enables Apple’s hardware-accelerated audio processing.
--enable-gpl & --enable-nonfree: Allows the use of GPL and non-free dependencies if you decide to add them (like x264 or x265).
--disable-shared & --enable-static: Builds a single, self-contained static executable.

Step 4: Compile and Install FFmpeg

Compile the source code using multiple CPU cores to speed up the process, then install the compiled binary.

Compile the code (replace $(sysctl -n hw.ncpu) with the number of CPU cores you want to allocate):
```
make -j$(sysctl -n hw.ncpu)
```
Install the binary to your system:
```
sudo make install
```

Step 5: Verify the Installation

After the installation completes, verify that FFmpeg is installed and that the VideoToolbox encoders are successfully enabled.

Check the FFmpeg version:
```
ffmpeg -version
```
List the available VideoToolbox encoders to confirm hardware acceleration is supported:
```
ffmpeg -encoders | grep videotoolbox
```

You should see output listing encoders such as h264_videotoolbox, hevc_videotoolbox, and prores_videotoolbox, confirming that your custom-built FFmpeg binary is ready to use hardware acceleration.