Python Path Resolution Using PYTHONPATH

When Python executes an import statement, it searches a specific sequence of directories to locate the requested module or package. This article explains how the PYTHONPATH environment variable interacts with Python's internal sys.path list, the exact order in which directories are evaluated during path resolution, and best practices for configuring custom module locations.

How Python Locates Modules: sys.path

At runtime, Python's import system relies on a list of directory strings stored in sys.path. When you run import my_module, the interpreter iterates through sys.path from left to right and imports the first match it encounters.

By default, Python constructs sys.path using three main sources:

  1. The script's directory: The directory containing the script used to invoke the Python interpreter (or the current working directory if invoked interactively).
  2. Standard Library paths: The locations where standard Python libraries are installed.
  3. Site-packages directories: The default locations for third-party packages installed via package managers like pip.

The Role of PYTHONPATH

PYTHONPATH is an operating system environment variable that allows users to prepend custom directories to Python's module search path without modifying the source code or altering the global Python installation.

When Python initializes, it checks for the existence of PYTHONPATH. If present, Python parses the variable and inserts the specified directory paths into sys.path immediately after the script's entry directory and before the standard library and site-packages.

The Path Resolution Order

The complete resolution hierarchy in sys.path typically follows this order:

  1. sys.path[0]: The directory containing the input script (or "" for an interactive session).
  2. PYTHONPATH entries: Custom directories defined in the environment variable, inserted in the order they appear.
  3. Standard library directories: Built-in modules and standard libraries.
  4. Site-packages: Third-party packages added by the site module.

Because PYTHONPATH is inserted before the standard library and site-packages, modules placed in a directory listed in PYTHONPATH take precedence over third-party packages and can potentially override standard library modules with identical names.

Setting and Using PYTHONPATH

PYTHONPATH uses the host operating system's standard path separator: a colon (:) on Unix-based systems (Linux, macOS) and a semicolon (;) on Windows.

Linux and macOS

To set the variable for a single execution:

PYTHONPATH=/path/to/custom_modules python3 main.py

To set it for the entire shell session:

export PYTHONPATH="/path/to/custom_modules:$PYTHONPATH"

Windows (Command Prompt and PowerShell)

In Command Prompt:

set PYTHONPATH=C:\path\to\custom_modules;%PYTHONPATH%
python main.py

In PowerShell:

$env:PYTHONPATH="C:\path\to\custom_modules;" + $env:PYTHONPATH
python main.py

Inspecting Path Resolution in Code

You can verify how PYTHONPATH affects resolution by inspecting sys.path directly inside a script:

import sys

print("Module search paths:")
for index, path in enumerate(sys.path):
    print(f"{index}: {path}")

If PYTHONPATH contains valid directories, they will appear at indices 1 through N (immediately following index 0).

Potential Pitfalls