PEP 517 and PEP 518 Build Backends in Python
Modern Python packaging relies on PEP 517 and PEP 518 to decouple
package building from installation, replacing the legacy
setup.py execution model with a standardized, declarative
configuration. Together, these standards allow developers to choose
specialized build backends—such as Flit, Hatchling, and
Meson-python—while tools like pip and build
act as frontends that orchestrate dependencies and invoke standardized
hooks to produce distribution packages.
The Legacy Problem:
Direct setup.py Execution
Historically, building a Python package required executing a
setup.py script, which implicitly mandated
setuptools as a pre-installed dependency. This created a
bootstrapping paradox: tools could not discover what a project needed to
build itself without first executing the project's code, leading to
environment contamination, inconsistent build environments, and an
inability to use alternative packaging tools.
PEP 518: Declaring Build Dependencies
PEP 518 solved the bootstrapping dilemma by introducing
pyproject.toml as a standard configuration file. Within
this file, the [build-system] table explicitly lists the
dependencies required to execute the build process before any code
runs:
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"A frontend like pip reads this section, creates an
isolated temporary environment, installs the specified build tools, and
only then initiates the build.
PEP 517: Standardizing Build Backends
While PEP 518 declares what tools are needed, PEP 517 defines how frontends interact with backends. It establishes a uniform Python API (a set of hooks) that any build engine must implement, such as:
build_wheel: Generates a binary distribution (.whl).build_sdist: Generates a source distribution (.tar.gz).get_requires_for_build_wheel: Dynamically reports additional dependencies needed for compilation.
Because frontends only interact with these standard hooks, they no longer care about the underlying engine used to create the files.
Common Modern Build Backends
By removing the monopoly of setuptools, PEP 517 and PEP
518 enabled the rise of specialized backends tailored for different
project needs:
1. Flit
(flit_core.buildapi)
Flit is designed for simplicity and pure-Python modules. It avoids
complex configuration, enforces the use of standard metadata, and does
not support C extensions. Its backend, flit_core, is
lightweight, fast, and ideal for standard libraries that do not require
custom build steps.
2. Hatchling
(hatchling.build)
Hatchling is the extensible backend powering the Hatch project manager. It supports advanced features such as dynamic versioning, editable installs, custom build hooks, and plugin architectures. Hatchling is widely used for modern application and library development where pure-Python packages require flexible metadata handling without legacy overhead.
3. Meson-python
(mesonpy)
Meson-python bridges Python packaging with the Meson build system. It
is engineered for high-performance projects containing compiled C, C++,
Cython, or Fortran extensions. Heavy scientific computing projects such
as NumPy and SciPy utilize mesonpy because it handles
complex native compilation tasks significantly faster and more reliably
than traditional Python packaging tools.
Summary of Advantages
The division of responsibilities introduced by PEP 517 and PEP 518 delivers several key advantages:
- Isolation: Builds occur in clean environments, preventing dependency conflicts with the host system.
- Flexibility: Developers can match their project requirements to the right backend—whether that means pure-Python simplicity (Flit), high extensibility (Hatchling), or compiled speed (Meson-python).
- Interoperability: Any conforming frontend can build any conforming project without custom workarounds.