Understanding BEP 33: BitTorrent DHT Scraping

BitTorrent Enhancement Proposal 33 (BEP 33) defines a standardized method for retrieving swarm health statistics—such as the number of seeders and leechers—directly from the Mainline Distributed Hash Table (DHT) without relying on centralized trackers. This article explains the background of BEP 33, how the DHT scrape mechanism works, its KRPC message structures, the use of Bloom filters for count estimation, and its impact on trackerless BitTorrent operations.

The Background of BEP 33

In traditional BitTorrent workflows, a client queries a centralized tracker’s “scrape” convention to check the number of active seeders and leechers for a torrent before initiating a download. As the BitTorrent ecosystem shifted toward trackerless operations via the Kademlia-based Mainline DHT (BEP 5), clients faced a limitation: the standard DHT get_peers query could discover active peer IP addresses, but it could not provide a lightweight, aggregate count of total swarm participants.

BEP 33 was introduced to bridge this gap. It provides a standardized protocol extension enabling nodes to query DHT peers for swarm metadata counts across multiple torrents efficiently.

How BEP 33 DHT Scraping Works

BEP 33 builds on BitTorrent’s KRPC protocol (Bencoded Remote Procedure Calls) by introducing a dedicated query type: scrape_hashes.

Instead of routing deep searches to find individual peer contact details, a client sends a scrape_hashes request to the DHT nodes responsible for the target info-hashes. The queried nodes respond with compact statistical data representing the active seeders and peers associated with those info-hashes.

The scrape_hashes Query

A querying node sends a KRPC message containing a list of target torrent info-hashes. The message structure includes:

• t (Transaction ID): A string representing the transaction identifier.
• y (Message Type): Set to q for query.
• q (Query Method): Set to scrape_hashes.
• a (Arguments): A dictionary containing:
  • id: The 20-byte node ID of the querying node.
  • info_hashes: A list of 20-byte strings, each representing a torrent info-hash.

The Response Structure and Bloom Filters

Handling raw lists of thousands of IP addresses simply to calculate peer counts creates significant network overhead. To solve this, BEP 33 utilizes Bloom filters—probabilistic, space-efficient data structures that estimate the number of unique items.

When a node replies to a scrape_hashes query, it returns a dictionary containing a files key mapped to the requested info-hashes. For each info-hash, the response typically includes:

• BFsd (Bloom Filter for Seeds): A compact binary Bloom filter containing hashed representations of known seed IP addresses and ports.
• BFpe (Bloom Filter for Peers): A compact binary Bloom filter containing hashed representations of known non-seed peer IP addresses and ports.
• seeders / downloaders (Optional): Direct integer estimates of active seeds and leechers if Bloom filters are not used or are supplemented.

By calculating the population count (the number of set bits) in the received Bloom filter, the querying client can mathematically estimate the total number of unique seeders and leechers in the swarm with minimal bandwidth usage.

Benefits of BEP 33

• Bandwidth Conservation: Clients can check the availability and health of hundreds of torrents simultaneously using tiny data payloads.
• Decentralized Swarm Health: BitTorrent clients and indexing tools can display accurate seeder and leecher counts for magnet links without querying traditional HTTP or UDP trackers.
• Privacy Enhancement: Using Bloom filters prevents the full list of IP addresses from being transmitted during a simple health check query.