Which Testing Frameworks Are Used in Libaom?

The libaom build system, which is the reference software library for the AV1 video codec, integrates specific testing frameworks to ensure code quality, performance, and stability across different platforms. This article provides an overview of the primary testing tools embedded within the libaom CMake build infrastructure, focusing on Google Test (GTest) for unit and integration testing, alongside the benchmarking utilities used to measure performance. Understanding these frameworks is essential for developers looking to contribute to or validate the AV1 reference encoder and decoder.

Google Test (GTest) Integration

The cornerstone of the libaom testing suite is Google Test (GTest). The CMake build system is configured to automatically download, configure, and compile GTest when testing is enabled.

• Unit Testing: Libaom utilizes GTest to verify the correctness of individual functions, particularly low-level pixel processing, transform blocks, and quantization algorithms.
• Integration and Codec Testing: Beyond simple unit tests, the framework runs large-scale conformance tests using reference vectors to ensure that the encoder and decoder adhere strictly to the AV1 specification.

WebM Test Data and Vector Validation

To complement the GTest framework, the libaom build system integrates a mechanism to download and manage external test vectors, often referred to as the WebM test data.

• Automated Downloads: When running the test suite, the build system checks for a local repository of test files. If missing, it fetches the required .ivf and .webm video files from the official Google Git repositories.
• Decode Integrity: These files are passed through the compiled decoder to verify that changes to the codebase do not introduce visual artifacts or decoding regressions.

Performance Benchmarking Tools

While GTest handles correctness, libaom also includes micro-benchmarks to track the execution speed of critical code paths, especially assembly optimizations (like AVX2, AVX-512, and NEON).

• Custom Vector Benchmarks: The build system compiles specific testing binaries (such as test_libaom) that can be executed with command-line flags to measure the clock cycles or execution time of specific kernel functions.
• Comparison Testing: These benchmarks allow developers to compare the performance of optimized assembly code directly against the C-language equivalent to ensure a measurable speedup.

Enabling Tests in the Build System

To utilize these integrated frameworks, developers must explicitly enable them during the CMake generation phase. By default, tests may be disabled to speed up compilation.

• The primary configuration flag used is -DENABLE_TESTS=1.
• Once configured, the entire testing suite can be executed using standard build tools, such as running make test or utilizing ctest within the build directory.