How Torrent Clients Locate Missing Data Files

When importing existing files into a new torrent—a process often used for cross-seeding or repairing broken downloads—a BitTorrent client uses the torrent’s metadata to map, scan, and cryptographically verify local files against the expected swarm payload. By comparing local file paths, file sizes, and block-level cryptographic hashes, the client accurately determines which files are complete, which pieces are corrupt or missing, and what data still needs to be downloaded from peers.

1. Reading the Torrent Metadata

Before checking the local storage, the client reads the .torrent file or retrieves metadata from the swarm via a magnet link. This metadata (the “info dictionary”) contains:

• The root folder structure and individual file paths.
• The exact size of each file in bytes.
• The defined piece size (typically ranging from 256 KB to 32 MB).
• A sequential list of SHA-1 (BitTorrent v1) or SHA-256 (BitTorrent v2) hashes for every piece.

2. Directory and File Matching

When you import existing data, you specify a target download directory. The client matches local content against the metadata:

• Path Alignment: The client searches the specified directory for subfolders and file names that match the file tree in the metadata.
• Size Pre-Check: The client queries the file system for the byte size of each located file. If a file is smaller than expected, the missing tail is flagged immediately. If a file is larger or completely missing, the client notes the discrepancy for the hashing phase.

3. Cryptographic Piece Verification (Force Recheck)

Once the files are mapped, the client performs a manual or automatic “Force Recheck” to find missing data at the sub-file piece level:

1. Reading File Blocks: The client sequentially reads raw binary data from the local storage in block sizes matching the torrent’s piece length.
2. Cross-File Piece Boundaries (BitTorrent v1): In BitTorrent v1, pieces are continuous across file boundaries. If File A ends mid-piece, the client combines the end of File A and the beginning of File B to generate the single piece for hashing.
3. Hash Calculation: The client generates a cryptographic hash for each local piece.
4. Comparison: It compares the generated hash with the piece hash stored in the metadata.
   • Match: The piece is marked as 100% complete and valid.
   • Mismatch or Missing File: The piece is marked as invalid, corrupted, or missing.

4. Marking Missing Blocks for Download

After verifying all pieces, the client updates its internal bitfield (a binary map representing owned pieces):

• Missing files or failed pieces are assigned a value of 0.
• Verified valid pieces are assigned a value of 1.

The client then transitions from checking to downloading mode, requesting only the specific missing or mismatched pieces from active peers rather than re-downloading the entire dataset.

5. BitTorrent v2 Improvements

Under the BitTorrent v2 specification, locating missing files is more precise. BitTorrent v2 uses per-file Merkle trees rather than global piece arrays across the entire torrent:

• Pieces never span across two different files.
• If a single file in a multi-file torrent is missing or modified, only the hash tree for that specific file fails verification.
• The client can immediately identify the exact missing file without invalidating adjacent, boundary-sharing pieces of complete files.