Python shutil Module: High-Level File Operations

Python's built-in shutil module provides a powerful suite of high-level utilities for managing files, directories, and file systems. While lower-level modules like os handle primitive operating system calls, shutil abstracts complex tasks into simple, readable functions. This guide explores the core utilities of shutil, covering file copying, recursive directory management, data archiving, and system inspection.

Copying Files and Directories

The shutil module offers several specialized functions for copying data, depending on whether file metadata needs to be preserved:

• shutil.copy(src, dst): Copies the content and permission bits of a file from source to destination.
• shutil.copy2(src, dst): Similar to copy(), but preserves additional metadata, such as creation and modification timestamps.
• shutil.copytree(src, dst): Recursively copies an entire directory tree. By default, the destination directory must not already exist, though modern Python versions support the dirs_exist_ok=True parameter to allow merging into existing directories.

import shutil

# Copy a single file preserving metadata
shutil.copy2('source.txt', 'backup.txt')

# Recursively copy an entire directory
shutil.copytree('source_folder', 'destination_folder', dirs_exist_ok=True)

Moving and Renaming Files

The shutil.move(src, dst) function handles moving files and directories across different paths or storage devices. If the destination is on the same filesystem, it acts as an atomic rename. If the destination resides on a different drive or partition, shutil automatically handles copying the data and deleting the original file, eliminating the need for manual fallback logic.

# Move a file or directory
shutil.move('data/records.csv', 'archive/records_2023.csv')

Removing Directory Trees

While os.rmdir() only removes empty directories, shutil.rmtree(path) deletes an entire directory tree recursively, removing all contained files and subfolders. This function is essential for cleanup scripts and automated build processes.

# Permanently delete a non-empty directory tree
shutil.rmtree('temp_cache')

Creating and Extracting Archives

The shutil module natively supports compressed archive formats, including ZIP, TAR, GZ, and BZ2, without requiring direct use of the zipfile or tarfile modules:

• shutil.make_archive(base_name, format, root_dir): Packages a directory into an archive file.
• shutil.unpack_archive(filename, extract_dir): Extracts the contents of an archive directly into the specified directory.

# Create a zip archive of a directory
shutil.make_archive('project_backup', 'zip', 'project_folder')

# Unpack a zip or tar archive
shutil.unpack_archive('project_backup.zip', 'extracted_folder')

System and Disk Utilities

Beyond file manipulation, shutil includes functions to query system environments and disk storage:

• shutil.disk_usage(path): Returns total, used, and free space on the drive containing the specified path.
• shutil.which(cmd): Locates the executable path of a command in the system PATH, mimicking the Unix which command.

# Check disk space (in gigabytes)
total, used, free = shutil.disk_usage("/")
print(f"Free space: {free // (2**30)} GB")

# Check if an executable exists
git_path = shutil.which('git')
print(f"Git executable: {git_path}")

shutil vs. os

The primary utility of shutil is abstraction. Where the os module requires developers to manually write recursive loops to copy or delete directory trees, shutil provides single-line solutions that handle operating system differences, permissions, and cross-device boundaries automatically.