TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Kademlia: A Peer-To-peer Information System Based on the XOR Metric (2002) [pdf]

191 pointsby liamzebedeeover 6 years ago

12 comments

bcaa7f3a8bbcover 6 years ago
I&#x27;m familiar to this paper because I was using eMule. eMule has a direct citation to this paper in its &quot;help&quot; window, otherwise, the decentralization of eD2k would not be possible. The protocol is old, but still impressive.<p>The year 2000 is special, within the same year, Gnutella, eDonkey and Freenet were all released its initial version, as if the P2P network &quot;emerged&quot; from the Internet. By 2004, eMule, I2P and BitTorrent (and BTW, Tor) were all here. At the very beginning, the first generation of P2P, Gnutella was suffering from its scalability problem - its simple unstructured, flooding protocol needs a number of O(n) queries, and becomes unstable once the users exceeded millions.<p>Then, Kademlia almost saved P2P. BitTorrent, I2P, eD2k and many other P2P protocols are all Kad-based. For example, in eD2k, Kad allowed it to have P2P resource discovery, and also an efficient decentralized keyboard search engine (although there is spam, but mostly works, and in BitTorrent we still don&#x27;t have it), all with only a O(log(n)) query, The P2P mechanism of I2P is also exclusively based on Kad.<p>The authors of this paper are the few people who have made great contributions to the Internet.
nostrademonsover 6 years ago
I love how people are rediscovering the previous generation of P2P software from 15-20 years ago. Reading through these papers was a huge part of my college procrastination; maybe now that cryptocurrencies give a way to provide economic incentives to those who contribute computing power, we might see a renaissance of some of these older ideas.
评论 #18713285 未加载
评论 #18712923 未加载
评论 #18712163 未加载
评论 #18722669 未加载
评论 #18714034 未加载
jMylesover 6 years ago
Kademlia is awesome and is, of course, the basis of libp2p-kad-dht and the DevP2P project that many decentralized projects are using at the moment.<p>At NuCypher, we found that, all the way to the core of Kademlia, there were some &quot;inter-planetary&quot; assumptions that, while both mindblowing and fit for many use-cases, didn&#x27;t fit our &quot;intra-planetary&quot; distributed notion of network topology.<p>If Kademlia interests you, you might also be interested in this overview of what we did and why:<p><a href="https:&#x2F;&#x2F;blog.nucypher.com&#x2F;why-were-rolling-our-own-intra-planetary-node-discovery-at-nucypher-beeb53018b0" rel="nofollow">https:&#x2F;&#x2F;blog.nucypher.com&#x2F;why-were-rolling-our-own-intra-pla...</a>
评论 #18713177 未加载
jonduboisover 6 years ago
A lot of cryptocurrency projects are using Kademlia but it&#x27;s prone to vulnerabilities such as eclipse attacks. Because peer connections are made according to deterministic rules (e.g. often based on unique Node IDs), the topology of the network can be analyzed and then the information can be exploited to disrupt the network.<p>There are solutions which involve making it hard to generate IDs (I&#x27;ve seen it done with node IDs generated using hash-based proof of work) but nowadays with all the cheap ASIC miners, you could compute these hashes much faster than any regular computer could; this means that node IDs need to be generated using top-of-the-range ASICs in order to make the network truly secure; this adds a lot of complexity when it comes to deploying a node. I don&#x27;t believe that any of the existing Kademlia-based cryptocurrencies currently require sufficiently complex node IDs.
tombertover 6 years ago
I wrote a video-sharing thing during a hackathon based on Kademlia. It&#x27;s one of my favorite algorithms and I found the paper to be easier to read than most papers based on algorithms. Here&#x27;s a video of a talk I gave on it:<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=byIFT8OzfX0" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=byIFT8OzfX0</a>
mohitmunover 6 years ago
Recently I figured Kademlia is how bittorrent achieves trackerless torrents. I used bittorrent-dht[0][1] to build simple decentralised ride hailing app(was able implement POC with end to end booking). I wonder what other modern problems can be solved by this technology<p>[0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;webtorrent&#x2F;bittorrent-dht" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;webtorrent&#x2F;bittorrent-dht</a><p>[1]: <a href="http:&#x2F;&#x2F;www.bittorrent.org&#x2F;beps&#x2F;bep_0005.html" rel="nofollow">http:&#x2F;&#x2F;www.bittorrent.org&#x2F;beps&#x2F;bep_0005.html</a>
woadwarrior01over 6 years ago
Over a decade ago, I was a part of a small team at a startup that built a p2p messenger called cSpace[1], I fondly remember reading the paper and implementing a Kademlia DHT for it. IIRC, the bittorrent DHT was the only open source Python implementation of it that we could find, it was using Twisted for networking and quite complicated, we decided to keep things simple and rewrite it ourselves. The resulting implementation[2] is quite simple and elegant, IMO.<p>[1]: <a href="https:&#x2F;&#x2F;github.com&#x2F;jmcvetta&#x2F;cspace" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jmcvetta&#x2F;cspace</a> (The only copy of it, I could find online) [2]: <a href="https:&#x2F;&#x2F;github.com&#x2F;jmcvetta&#x2F;cspace&#x2F;tree&#x2F;master&#x2F;cspace&#x2F;dht" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;jmcvetta&#x2F;cspace&#x2F;tree&#x2F;master&#x2F;cspace&#x2F;dht</a>
loegover 6 years ago
(2007), if not earlier, FWIW. I used this protocol in a college final project (&quot;make a botnet&quot;) years ago :-).
评论 #18712397 未加载
评论 #18712361 未加载
cfover 6 years ago
Interestingly, Kademlia ended up being a major component in lots of botnets for a time like <a href="https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Storm_botnet" rel="nofollow">https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Storm_botnet</a>
miguelrochefortover 6 years ago
Has any progress in the P2P a been made since Kademlia?<p>I&#x27;ve been searching for a while, for a DHT (actually a distributed graph database) that self optimizes based on usage. Basically, I want data that&#x27;s frequently accessed together to get close to eachother and to nodes that frequently access it.
评论 #18716607 未加载
new12345over 6 years ago
I don&#x27;t understand inspite of having working p2p protocol why we don&#x27;t yet have an good p2p alternative to dropbox that just works without any setup hassles. Closest working thing I know of is BitSync but that too requires an mediator server.
评论 #18713410 未加载
评论 #18713828 未加载
评论 #18718118 未加载
评论 #18714360 未加载
dmouratiover 6 years ago
Attended a &quot;Papers we Love&quot; presentation on this paper last year.