How to Compile FFmpeg with libaom AV1 on Windows

Compiling FFmpeg with the libaom AV1 library on Windows enables custom, highly optimized builds featuring the reference AV1 video encoder. This guide provides a straightforward, step-by-step walkthrough using MSYS2 and the MinGW-w64 toolchain to fetch required dependencies, build and install libaom, configure the FFmpeg source code with the --enable-libaom flag, and compile the final standalone binaries.

Prerequisites: Setting Up MSYS2

The most reliable way to build native Windows binaries of FFmpeg is via MSYS2.

1. Download and install MSYS2 from its official site.

2. Launch the MSYS2 MINGW64 shell from your Start menu (do not use the standard MSYS environment).

3. Update the package database and core packages by running:

   pacman -Syu

   Restart the terminal if prompted, then run the command once more to ensure all base packages are current.

Step 1: Install Build Tools and Dependencies

Install the necessary compilers, build systems, and libraries required to build both libaom and FFmpeg:

pacman -S --needed \
    base-devel \
    mingw-w64-x86_64-toolchain \
    mingw-w64-x86_64-cmake \
    mingw-w64-x86_64-ninja \
    mingw-w64-x86_64-nasm \
    mingw-w64-x86_64-yasm \
    git

Step 2: Build and Install libaom

libaom provides the encoder (libaom-av1) required by FFmpeg. Build it as a static library to make distribution easier.

1. Clone the libaom repository and create a build directory:

   git clone --depth 1 https://aomedia.googlesource.com/aom
   cd aom
   mkdir build_aom && cd build_aom

2. Configure the build using CMake:

   cmake .. -G Ninja \
       -DCMAKE_BUILD_TYPE=Release \
       -DBUILD_SHARED_LIBS=OFF \
       -DENABLE_DOCS=OFF \
       -DENABLE_EXAMPLES=OFF \
       -DENABLE_TESTS=OFF \
       -DENABLE_TOOLS=OFF \
       -DCMAKE_INSTALL_PREFIX=/mingw64

3. Compile and install:

   ninja
   ninja install
   cd ../..

This command installs the header files and libaom.a directly into the MSYS2 MinGW-w64 directory structure, allowing FFmpeg's configuration script to detect it automatically.

Step 3: Download and Configure FFmpeg

1. Clone the FFmpeg source repository:

   git clone --depth 1 https://git.ffmpeg.org/ffmpeg.git
   cd ffmpeg

2. Run the configure script. Make sure to include --enable-libaom along with other standard flags for standalone Windows builds:

   ./configure \
       --prefix=/mingw64 \
       --pkg-config-flags="--static" \
       --extra-cflags="-static" \
       --extra-ldflags="-static" \
       --enable-gpl \
       --enable-version3 \
       --enable-static \
       --disable-shared \
       --enable-libaom

If the configuration completes without errors, your build environment successfully detected the libaom headers and libraries.

Step 4: Compile FFmpeg

Execute the build using all available CPU cores:

make -j$(nproc)

The compilation process will take several minutes depending on your hardware.

Step 5: Verify the Build

Once compilation completes, test the newly compiled binary in your build folder:

./ffmpeg.exe -encoders | grep libaom

You should see output similar to:

 V..... libaom-av1           libaom AV1 (codec av1)

Your compiled ffmpeg.exe and ffprobe.exe binaries are located directly in the current directory, fully capable of encoding AV1 video using libaom.