TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Show HN: Fast Rust BitTorrent Tracker

3 pointsby greatest-apeabout 5 years ago

1 comment

greatest-apeabout 5 years ago
Author here.<p>I wrote a BitTorrent tracker in Rust a few years ago. A while ago, I found out about the dashmap crate (providing a concurrent hashmap) and thought that I could use it for a new implementation, which could be both performant and simple. I tried using it, but due to blocking, I had a hard time reaching good performance. My realization was that since BitTorrent tracker requests are highly skewed towards popular torrents, even a map which only locks on accesses to the same key would not scale well when used like this.<p>I experimented and settled on a design with many threads each opening a socket with SO_REUSEPORT and using mio to wait for requests. The requests are read, parsed and sent through a crossbeam channel to request workers, which collect a number of them before locking the mutexes guarding shared state and doing work required to generate responses. These are sent back to the socket workers, which encode them and send them back over the network.<p>This solution is quite performant and during heavy load testing, I couldn’t even get a lone request worker to reach full CPU utilization. (Adding more would likely not bring much improvement anyway since they would block each other most of the time). Results in terms of responses per second indicate that aquatic scales a lot better with more threads than the established project opentracker, being able to reach more than twice the throughout at best, at the price of using many more threads. (Benchmarks are available in the README file.)<p>I’d like to thank (in particular) the authors of mio, crossbeam, hashbrown and indexmap for making this fun project possible.