How ut_metadata Protocol Transfers Torrent Metadata

The ut_metadata extension is a BitTorrent enhancement that allows clients to fetch the metadata (the .torrent file’s info dictionary) directly from swarm peers rather than relying on a centralized web server. By establishing an extension handshake, calculating the total payload size, segmenting the raw bencoded dictionary into fixed 16 KiB pieces, and validating the final assembly against the magnet link’s info-hash, peers can efficiently share and reconstruct the complete torrent file on the fly.

Extension Handshake and Negotiation

The ut_metadata extension operates on top of the BitTorrent Extension Protocol (BEP 10). When two peers connect, they exchange an extension handshake dictionary containing the key m, which maps supported extension names to local message IDs.

During this handshake, a peer advertising the metadata will include: * The Extension ID: An arbitrary integer mapped to "ut_metadata". * metadata_size: The total byte size of the raw info dictionary.

Once this handshake is complete, both clients know which message ID to route metadata requests to and the exact size of the payload to be transferred.

Dividing the Metadata into Pieces

Unlike standard BitTorrent payload data, which can have variable piece sizes, ut_metadata standardizes metadata segmentation to fixed blocks:

1. Chunk Size: The raw, bencoded info dictionary is divided into chunks of 16 KiB (16,384 bytes) each.
2. Piece Indexing: The total number of pieces is calculated as ceil(metadata_size / 16384).
3. Numbering: Pieces are indexed sequentially starting from 0 up to N - 1. The final piece contains the remainder bytes and may be smaller than 16 KiB.

The Message Exchange Cycle

All communication occurs using bencoded message headers appended with raw data, categorizing communication into three standard message types:

• Request (msg_type: 0): A client wishing to download metadata sends a request dictionary specifying the target piece index.
• Data (msg_type: 1): The serving peer responds with a message that includes a bencoded header detailing the msg_type, piece index, and total_size, immediately followed by the raw binary chunk corresponding to that piece.
• Reject (msg_type: 2): If a peer does not possess the requested metadata or cannot serve it, it sends a reject message with the requested piece index so the requesting client can query another peer.

Reassembly and Verification

Because the client downloads pieces out of order and across multiple peers, verification is crucial before proceeding with the main file download:

1. Buffer Assembly: The client writes each incoming 16 KiB slice into a pre-allocated memory buffer based on its piece index and offset (piece * 16384).
2. Cryptographic Validation: Once all pieces are received, the client computes the SHA-1 hash (or SHA-256 in BitTorrent v2) across the entire assembled byte array.
3. Info-Hash Matching: The calculated hash is compared against the info-hash specified in the original magnet link. If the hashes match, the client successfully parses the bencoded data to obtain the file list, tracker list, and piece hashes needed to begin the actual content download. If the hash does not match, the assembled data is discarded as corrupted or malicious.