Compile FFmpeg with libaom AV1 Encoder on Windows

This guide provides a step-by-step walkthrough on how to compile FFmpeg with the libaom AV1 video encoder on Windows. By leveraging MSYS2 and the MinGW-w64 toolchain, you will learn how to set up the build environment, install the necessary dependencies, download the source code, and successfully compile a custom FFmpeg binary with native AV1 encoding capabilities.

Step 1: Install MSYS2

MSYS2 provides a terminal environment and the package manager (pacman) needed to install build tools on Windows.

  1. Download and run the MSYS2 installer from the official MSYS2 website.
  2. Follow the installation wizard instructions.
  3. Once installed, open the MSYS2 MinGW 64-bit terminal from your Start Menu. Do not use the default MSYS terminal, as the MinGW 64-bit terminal is required to compile 64-bit Windows binaries.

Step 2: Update the Package Database and Core Packages

In the MSYS2 MinGW 64-bit terminal, run the following command to update the package database and core system packages:

pacman -Syu

If the terminal asks you to close it, close the window and reopen the MSYS2 MinGW 64-bit terminal, then run the command again to ensure everything is fully updated:

pacman -Su

Step 3: Install Build Tools and Dependencies

To compile FFmpeg and link it with the libaom encoder, you need to install the GCC compiler, build utilities, and the libaom development package. Run the following command:

pacman -S mingw-w64-x86_64-toolchain \
          mingw-w64-x86_64-cmake \
          mingw-w64-x86_64-yasm \
          mingw-w64-x86_64-nasm \
          mingw-w64-x86_64-aom \
          git \
          make \
          pkg-config \
          diffutils

When prompted to select members of the toolchain, press Enter to install all of them.

Step 4: Clone the FFmpeg Source Code

Clone the official FFmpeg git repository to your local directory:

git clone https://git.ffmpeg.org/ffmpeg.git ffmpeg-source
cd ffmpeg-source

Step 5: Configure the Build

Configure the compilation settings to enable the libaom encoder. You must enable GPL and use the --enable-libaom flag. Run the following configure command:

./configure \
  --prefix=/usr/local \
  --enable-gpl \
  --enable-nonfree \
  --enable-libaom \
  --disable-shared \
  --enable-static \
  --extra-libs=-lpthread \
  --extra-libs=-lm

Note: --disable-shared --enable-static ensures that you get a single, self-contained executable file (ffmpeg.exe) that does not rely on external DLLs.

Step 6: Compile and Install FFmpeg

Once the configuration is complete without any errors, compile the source code. You can use multiple CPU threads to speed up the compilation process by appending -j followed by the number of CPU cores (e.g., -j8 for 8 cores, or -j$(nproc) to use all available cores):

make -j$(nproc)

After the compilation finishes successfully, install the binary to your MSYS2 local directory:

make install

Step 7: Verify the Installation

Your newly compiled ffmpeg.exe binary will be located in the /usr/local/bin folder within MSYS2, which maps to C:\msys64\usr\local\bin in Windows.

To verify that the binary was built correctly and supports the libaom AV1 encoder, run:

/usr/local/bin/ffmpeg -version

Look for --enable-libaom in the configuration output.

To confirm that the AV1 encoder is ready for use, run:

/usr/local/bin/ffmpeg -encoders | grep aom

You should see libaom-av1 listed as an available encoder. You can now use this binary to encode videos to AV1 format on Windows.