How pip-tools Generates Pinned requirements.txt

This article explains how the Python utility pip-tools bridges the gap between flexible dependency definitions and deterministic builds by generating pinned requirements.txt lockfiles. You will learn the mechanics behind the core command pip-compile, how it resolves transitive dependencies, the distinction between source requirements and lockfiles, and how to maintain reproducible Python environments effectively.

The Separation of Concerns: .in vs .txt

Traditional Python package management often forces developers to choose between loosely defined dependencies that easily break over time and hard-pinned lists that obscure which packages are top-level requirements. pip-tools solves this by decoupling your intended dependencies from the resolved environment using two distinct files:

• requirements.in: The source of truth containing abstract, top-level dependencies. Developers specify only the direct packages they need, often using loose version constraints (e.g., requests>=2.25.0 or Flask).
• requirements.txt: The generated lockfile containing concrete, fully pinned versions (package==1.2.3) of all direct and indirect (transitive) dependencies required to run the project.

How pip-compile Resolves and Pins Dependencies

The primary engine behind pip-tools is the pip-compile command. When you run pip-compile requirements.in, the tool executes a multi-step resolution process:

1. Parsing Direct Dependencies: pip-compile reads requirements.in to identify the explicit packages and any specified version bounds.
2. Traversing Transitive Trees: The tool connects to the package index (typically PyPI) and fetches the metadata for each top-level requirement to determine its underlying dependencies. It recursively examines every sub-dependency.
3. Conflict Resolution: Using a backtracking resolver, pip-compile finds a single, compatible version for every package in the dependency graph. If two packages require conflicting versions of a shared library, the command fails and alerts you to the conflict.
4. Pinning Exact Versions: Once a compatible graph is calculated, pip-compile locks every package to an exact release version using the == operator.
5. Emitting Annotated Output: The tool writes the resolved graph to requirements.txt. By default, it annotates each package with comments indicating which parent dependency requested it (e.g., # via requests).

Generating Cryptographic Hashes

To protect against supply-chain attacks and ensure packages have not been tampered with on the index, pip-compile can generate SHA-256 integrity hashes:

pip-compile --generate-hashes requirements.in

When run with this flag, pip-compile fetches the checksums for every target distribution (wheels and source tarballs) and appends them to the generated requirements.txt. When installing with pip install -r requirements.txt, pip verifies that downloaded artifacts match these exact hashes.

Updating and Synchronizing Dependencies

pip-tools also provides deterministic workflows for maintenance:

• Upgrading Packages: By default, running pip-compile preserves existing versions in requirements.txt if they satisfy the .in constraints. To update dependencies, you run pip-compile --upgrade to bump all packages to their latest compatible versions, or pip-compile --upgrade-package <pkg> to bump a specific library.
• Applying the Lockfile (pip-sync): While pip install -r requirements.txt installs packages, it does not remove unused dependencies. The companion command pip-sync compares the virtual environment to requirements.txt, installing missing packages, updating modified versions, and uninstalling any unlisted packages to guarantee strict environment synchronization.