How to Recursively Scan Directories in C++

This article provides a comprehensive overview of directory iterators in C++, focusing specifically on how to use std::filesystem::recursive_directory_iterator to traverse nested file systems. Introduced in C++17, the <filesystem> library standardizes file and path manipulation, eliminating the need for platform-specific APIs. Readers will learn the mechanics behind recursive directory iteration, see practical code examples, and discover essential techniques for handling file permissions, symbolic links, and error states.

Understanding Directory Iterators in C++

The C++ <filesystem> library introduces two primary iterator types for traversing folders: std::filesystem::directory_iterator and std::filesystem::recursive_directory_iterator.

• directory_iterator: Iterates only over the contents of a single directory, ignoring subdirectories.
• recursive_directory_iterator: Iterates over a directory and automatically descends into subdirectories, enabling a full depth-first traversal of an entire file tree without requiring manual recursion logic.

Both iterators yield std::filesystem::directory_entry objects, which contain paths and cached metadata attributes such as file size, modification time, and file type.

Implementing a Recursive File System Scan

To recursively scan a directory, instantiate a std::filesystem::recursive_directory_iterator pointing to the target path and iterate through it using a standard range-based for loop.

#include <iostream>
#include <filesystem>

namespace fs = std::filesystem;

void scanDirectory(const fs::path& targetPath) {
    if (!fs::exists(targetPath) || !fs::is_directory(targetPath)) {
        std::cerr << "Invalid path provided.\n";
        return;
    }

    for (const auto& entry : fs::recursive_directory_iterator(targetPath)) {
        const auto& path = entry.path();
        
        if (entry.is_regular_file()) {
            std::cout << "[FILE] " << path.string() 
                      << " (" << entry.file_size() << " bytes)\n";
        } else if (entry.is_directory()) {
            std::cout << "[DIR]  " << path.string() << "\n";
        }
    }
}

int main() {
    scanDirectory("./sample_directory");
    return 0;
}

Controlling Traversal and Recursion Depth

The recursive_directory_iterator provides member functions to monitor and manipulate the traversal process during runtime:

• depth(): Returns the current recursion depth relative to the starting directory.
• disable_recursion_pending(): Instructs the iterator not to descend into the directory entry currently pointed to, skipping that specific branch.
• pop(): Immediately exits the current subdirectory level and moves back up to the parent directory.

for (auto it = fs::recursive_directory_iterator(targetPath); 
     it != fs::recursive_directory_iterator(); ++it) {
    
    // Prevent entering directories named "node_modules" or ".git"
    if (it->is_directory() && it->path().filename() == "node_modules") {
        it.disable_recursion_pending();
        continue;
    }

    std::cout << "Depth " << it.depth() << ": " << it->path().string() << '\n';
}

Handling Permissions and Symlinks with Iteration Options

File systems frequently contain protected directories or symbolic link cycles that can cause traversals to fail or loop indefinitely. C++ provides std::filesystem::directory_options bitmask flags to manage these behaviors:

1. directory_options::skip_permission_denied: Instructs the iterator to ignore directories where read permission is denied instead of throwing a std::filesystem_error.
2. directory_options::follow_directory_symlink: Enables following symbolic links to other directories (disabled by default to prevent infinite cycles).

fs::directory_options options = fs::directory_options::skip_permission_denied;

for (const auto& entry : fs::recursive_directory_iterator(targetPath, options)) {
    std::cout << entry.path().string() << '\n';
}

Error Handling with std::error_code

For production environments where exceptions are undesirable or prohibited, the iterator supports an overload accepting a std::error_code reference. This captures runtime errors—such as vanishing files during iteration—without throwing exceptions:

std::error_code ec;
fs::recursive_directory_iterator it(targetPath, options, ec);

if (ec) {
    std::cerr << "Initialization error: " << ec.message() << '\n';
    return;
}

while (it != fs::recursive_directory_iterator()) {
    std::cout << it->path().string() << '\n';
    it.increment(ec);
    
    if (ec) {
        std::cerr << "Iteration error: " << ec.message() << '\n';
        ec.clear(); // Clear the error to continue processing
    }
}

By leveraging std::filesystem::recursive_directory_iterator, modern C++ applications gain a cross-platform, robust, and highly configurable mechanism for deep directory traversal with minimal overhead.