Understanding BEP 9 BitTorrent Metadata Exchange

This article provides an overview of BitTorrent Enhancement Proposal 9 (BEP 9), the official specification governing how peers exchange torrent metadata directly across a peer-to-peer network. It explains the foundational role BEP 9 plays in supporting magnet links, how it utilizes the BitTorrent Extension Protocol, its message structures and chunking mechanisms, and the process clients use to verify metadata integrity before beginning a download.

Purpose and Role in Magnet Links

BEP 9 specifies an extension that enables BitTorrent clients to retrieve the info dictionary—the core metadata file containing file names, sizes, and piece hashes—directly from swarm peers rather than downloading a standalone .torrent file from a centralized web server. This specification is the fundamental mechanism that allows magnet links to function, as a magnet link typically contains only the cryptographic hash (info_hash) of the metadata rather than the metadata itself.

Integration with BEP 10

BEP 9 builds on top of the BitTorrent Extension Protocol (BEP 10). When two peers establish a connection: 1. They exchange extension handshakes defined by BEP 10. 2. A peer supporting BEP 9 advertises the extension name ut_metadata along with an assigned extension message ID. 3. The handshake includes the metadata_size integer key, which communicates the total size of the torrent metadata in bytes.

Chunking Mechanism

Because torrent metadata files can be large, BEP 9 mandates dividing the metadata into standardized 16 KiB (16,384 bytes) blocks. The total number of pieces is derived by dividing the metadata_size by 16,384 and rounding up. Peers request and share these individual blocks independently, allowing metadata to be transferred quickly from multiple peers simultaneously.

Message Types

BEP 9 defines three message types exchanged as bencoded dictionaries within the BEP 10 extension wrapper:

• Request (msg_type: 0): Sent by a client to request a specific metadata piece by index. The message payload specifies the piece index integer.
• Data (msg_type: 1): Sent in response to a valid request. It includes the piece index and the total_size of the metadata. The raw binary piece data is appended immediately after the bencoded dictionary header.
• Reject (msg_type: 2): Sent if a peer cannot or will not fulfill a request (e.g., if the peer does not yet have the complete metadata or is rate-limiting the requester).

Integrity and Verification

Once a client successfully downloads all 16 KiB pieces, it concatenates them in order and calculates the SHA-1 cryptographic hash of the entire payload. The client compares this result against the info_hash obtained from the initial magnet link. If the hashes match, the metadata is authenticated, and the client can parse the file structure, allocate disk space, and proceed to download the actual payload data from the swarm.