Python Pip Dependency Resolution and Conflicts

Python’s package installer, pip, manages complex environments by recursively parsing package requirements and automatically reconciling version mismatches. Since the release of version 20.3, pip has used a backtracking resolver powered by the resolvelib library, moving away from its legacy greedy approach. This article examines how pip discovers recursive dependencies, traverses the dependency tree, resolves conflicting constraints via backtracking, and handles unresolvable dependency conflicts.

The Recursive Discovery Process

When a package is installed using pip install, the resolver evaluates the requested package's metadata (found in distribution wheels or source packages via pyproject.toml, setup.cfg, or setup.py).

1. Root Inspection: pip parses the primary package's direct dependencies and records their version specifiers.
2. Recursive Traversal: For each required dependency, pip downloads package metadata from the index (such as PyPI) without downloading the entire package archive if metadata is available. It extracts downstream dependencies, continuing this process iteratively until all branches of the dependency tree terminate.
3. Graph Construction: The result is a Directed Acyclic Graph (DAG) representing the complete set of required direct and transitive packages along with their specified version ranges.

The Backtracking Resolution Algorithm

Older versions of pip used a naive, "first-seen-wins" approach. If Package A required foo==1.0 and Package B required foo==2.0, pip would install whichever was encountered first, leaving the environment broken.

Modern pip uses a backtracking algorithm that evaluates the dependency tree globally:

• Constraint Accumulation: pip gathers every constraint applied to a specific package across all dependencies. For example, if Package A requires urllib3>=1.26.0,<2.0.0 and Package B requires urllib3>=1.21.1, the effective constraint becomes urllib3>=1.26.0,<2.0.0.
• Candidate Selection: pip attempts to select the newest available version of a package that satisfies all currently known constraints.
• Backtracking on Failure: If selecting a specific version of a package leads to a conflict downstream (for instance, a transitive dependency introduces a new constraint that invalidates the chosen version), pip reverses its decision. It discards the incompatible candidate and evaluates the next viable candidate version higher up in the dependency tree.

Resolving Incompatible Version Conflicts

When constraints are mutually exclusive, no valid set of packages can satisfy the environment. For example:

• Package X requires requests<2.25.0
• Package Y requires requests>=2.28.0

Because no release of requests can satisfy both conditions simultaneously, pip exhausts all possible backtracking combinations across the dependency tree. Once all candidate combinations fail, pip aborts the installation entirely to avoid corrupting the environment. It outputs a ResolutionImpossible error detailing:

• The conflicting packages.
• The exact dependency paths that introduced the conflicting constraints.
• The specific version ranges demanded by each dependent package.

Controlling Resolver Behavior

Users can manage how pip resolves dependencies through specific flags:

• --use-deprecated=legacy-resolver: Temporarily reverts to the pre-20.3 resolver behavior (not recommended, as it allows silently broken environments).
• Constraints Files (-c constraints.txt): Enforces global limits on specific package versions across an entire dependency tree without forcing them to be installed directly unless required by another package.
• --no-deps: Skips dependency resolution entirely, installing only the explicitly named target package.