Ask HN: What are some examples of “decentralized discoverability?”

I&#x27;ve heard about CommonCrawl [0] a few times here and it seems interesting to think of building a search engine around it. Or, maybe, a boilerplate search engine that people could build upon. Maybe someone will even reply with an example :) I see they have a big list of projects [1] and honestly this might already be in there. It&#x27;s fun to imagine a whole bunch of specialized search engines, like one that&#x27;s great at finding product reviews, one for software development, one for recipes, and so on.<p>I think the powerful aspect of a distributed&#x2F;mirrored index is that the data is public, so anyone can build a UI or a product around it to offer their unique spin.<p>From a more cynical angle, I also think it&#x27;s likely that people still tend to gather around a few &quot;popular&quot; ones of anything like this. The web3 stuff seemed to do this. It happened with torrent trackers too. Eventually there&#x27;s an option that is &quot;base + X&quot; where X is enticing enough to attract a crowd, and then once that crowd is big enough, that popular thing can pivot into a centralized thing, and the cycle repeats...<p>0: <a href="https:&#x2F;&#x2F;commoncrawl.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;commoncrawl.org&#x2F;</a><p>1: <a href="https:&#x2F;&#x2F;commoncrawl.org&#x2F;the-data&#x2F;examples&#x2F;" rel="nofollow">https:&#x2F;&#x2F;commoncrawl.org&#x2F;the-data&#x2F;examples&#x2F;</a>

You can use links, webrings, and word of mouth. You can crawl (for non-web this may require quite specialized tooling), scrape, and compile a database for search (which can be done locally using whatever tools you prefer or even a full-blown webserver like gigablast or YaCy); and, to some degree, these practices themselves can be further decentralized and shared. Scaling is hard, especially if users expect immediate results or there are adversarial nodes to contend with.<p>Ideally, we could programmatically share feeds of curated and tagged information with each other that can be nested, mixed, or further shaped. Leveraging our trust in others (and their trust in others by proxy), which can be similar to some problems in distributed governance, is a p2p holy grail, imho.

BitTorrent had a protocol extension for DHTs, as a replacement for the trackers that are required by default: <a href="https:&#x2F;&#x2F;www.bittorrent.org&#x2F;beps&#x2F;bep_0005.html" rel="nofollow">https:&#x2F;&#x2F;www.bittorrent.org&#x2F;beps&#x2F;bep_0005.html</a><p>So you still need a central web server to serve the .torrent file, but peers are found without centralisation. But I’m sure with effort that could have been generalised further.<p>There’s also IPFS (<a href="https:&#x2F;&#x2F;ipfs.tech" rel="nofollow">https:&#x2F;&#x2F;ipfs.tech</a>), which uses content addressability to try to be even more decentralised than the normal web (without any cryptocurrency shenanigans, at least last I checked).<p>I don’t think either took off massively though.

Using bittorrent in a really decentralized way: <a href="https:&#x2F;&#x2F;github.com&#x2F;boramalper&#x2F;magnetico&#x2F;" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;boramalper&#x2F;magnetico&#x2F;</a>

<a href="https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Kademlia" rel="nofollow">https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Kademlia</a>

OpenSea was popularized because the NFT minting functionality cost nothing for the artist. So all costs for the blockchain part was passed to the consumer. I believe this was the largest reason it became popular. Blockchain still doesn’t scale well, layer 2 is getting pretty mature though and with Eth Merge happening hopefully things will get significantly better in the next few years.

This reminds me a little of the cuban offline network (<a href="https:&#x2F;&#x2F;www.fastcompany.com&#x2F;3048163&#x2F;in-cuba-an-underground-network-armed-with-usb-drives-does-the-work-of-google-and-youtube" rel="nofollow">https:&#x2F;&#x2F;www.fastcompany.com&#x2F;3048163&#x2F;in-cuba-an-underground-n...</a>)