Re-decentralizing the Web, for good this time

It puzzles me that the linked data future is still discussed, as if we didn&#x27;t already try it, and didn&#x27;t already discover that developers dislike arcane RDF standards and the academic-rooted designers of the specifications have a terrible track record of solving real-world problems. And that now they&#x27;re presenting linked data as some critical component of the decentralized web while skipping out on the debates that everyone else in the space is having - like whether decentralization can be fast, or how to ensure data authenticity, or whether a &#x27;local server &#x2F; pod&#x27; can be built that doesn&#x27;t get hosed by hole-punching through a home Comcast connection.<p>Instead, it&#x27;s just &#x27;what about old-fashioned websites, plus lots of xml schema and long spec documents&#x27;? It just tastes like a rehash of Berners-Lee&#x27;s existing &#x27;5-star open data&#x27; schpiel ( <a href="https:&#x2F;&#x2F;5stardata.info&#x2F;en&#x2F;" rel="nofollow">https:&#x2F;&#x2F;5stardata.info&#x2F;en&#x2F;</a> ) but now with the billing that it&#x27;ll fix the internet. 5-star open data has been around for years now, and, well, the linked data future isn&#x27;t here. When&#x27;s the last time you consumed RDF in an application?

I think the part ignored by so many is the need to decentralize the computers into the home. I&#x27;m not talking meshes or shared resources. For the majority of use cases, we don&#x27;t need distributed storage, compute, etc. Just start making these self-hosted &quot;servers&quot;, &quot;data pods&quot;, etc as easy to install as desktop software and make it clear that they are inaccessible when the computer is off. People that aren&#x27;t already will gravitate towards at least one always-on machine in their house. Modern societies have reasonable upload speeds and electricity&#x2F;network uptime to support it. Sure, things like ISP firewall&#x2F;NAT and dynamic IPs are a bit of a barrier, but you can have volunteers help with relays.<p>For example, I can easily fire up a Tor onion service on my never-turns-off home desktop computer and reach my stuff from anywhere. Why can&#x27;t I reach my friends&#x27; stuff the same way? Because, to use business-speak, there&#x27;s nothing &quot;turnkey&quot;. It&#x27;s something I&#x27;ve been pondering and working on. Sure, the bigger players may have to be in DCs, have more stringent uptime requirements, and distribute their bandwidth&#x2F;workload more. But for most of us, desktop software and web-of-trust style connections could go a long way so long as the front of the software has a FB feel (e.g. a feed, messages, etc). We can tackle discovery, searching, aggregation, offloading, etc later.

The W3C has a proven track record to produce overengineered shit when it comes to &quot;the semantic web&quot;.<p>Just look at ActivityPub. It&#x27;s essentially OStatus but instead of XML we slapped namespaces on JSON, wrote a bunch of overly complex preprocessing procedures so that everyone can output just the way they want[1] and still made half the spec ambiguous enough[2] that implements essentially follow the one rule that matters, maintain compatibility with Mastodon.<p>[1]: <a href="https:&#x2F;&#x2F;www.w3.org&#x2F;TR&#x2F;json-ld-api&#x2F;#algorithm-5" rel="nofollow">https:&#x2F;&#x2F;www.w3.org&#x2F;TR&#x2F;json-ld-api&#x2F;#algorithm-5</a><p>[2]: <a href="https:&#x2F;&#x2F;please-just-end.me&#x2F;ap.html#block-activity-outbox" rel="nofollow">https:&#x2F;&#x2F;please-just-end.me&#x2F;ap.html#block-activity-outbox</a> (domain name relevant to content)

&quot;In order to regain freedom and control over the digital aspects of our lives&quot;<p>Nothing proves his point more than:<p><pre><code> &lt;script src=&quot;&#x2F;&#x2F;www.google-analytics.com&#x2F;analytics.js...</code></pre>

I skimmed this and see two big problems.<p>First, this idealistic idea that &quot;we&quot; are going to take back our data. Who is this we? Only the smart, high-agency people who have time to spare. The commercial web is increasingly tuned to the normal user, who is low-agency and easily led around. Who will win a battle of user acquisition and retention? Facebook or the rebels? Facebook of course. So any solutions proposed here are just for a tiny percentage of users who will then be isolated from the real and useful social networks. Or more realistically use both.<p>Or maybe if the infrastructure is built, a layer of savvy entrepreneurs can emerge to monetize it? I&#x27;m thinking of reaganemail, selling an anti-google email account to the AM radio crowd.<p>Second, the idea of somehow eliminating censorship. De facto censorship will always exist, even if you sugar coat it as Twitter has tried - &quot;your content is still there, but only if someone explicitly looks for it&quot;. Any platform without censorship will just be flooded by every marketer and political zealot, for starters.<p>Also, I think he is conflating filter bubbles with centralization. Without centralization, wouldn&#x27;t we still have filter bubbles as people self-select into their online communities?

I&#x27;ve read a bit about Solid in the past, but never quite understood how it will handle different data models. Does it force social data to all look the same (as in, have a predefined set of fields)? If not, how do apps built on it interoperate?<p>Don&#x27;t get me wrong, I&#x27;m all for projects like this. I think it&#x27;s wonderful. I just never really got how the apps will work with the same data without being forced into a particular data model (which seems like it would limit what you could do).

The web is decentralized, and we already have control of our data. The problem is, people keep giving it away.<p>I&#x27;m fairly confident that 98% of the population of the earth doesn&#x27;t give a crap that their data is collected, or that they don&#x27;t &quot;control&quot; it. This whole &quot;decentralized web&quot; thing is just privacy nerds trying to convince us that we need this, when really no regular consumer is asking for it.

Technology won&#x27;t help with this, regulation will.<p>We have parallels from other platforms - specifically the fixed and mobile phone networks.<p>There used to be monopolies in local phone service. There were new competitors, but to change provider, you had to change phone number.<p>Even changing cell phone provider required a number change.<p>This obviously had strong network effects pulling you to stay with your provider. You had to tell _everyone_ in your extended network where to find you and have them update all of their business records when you changed from one carrier to another.<p>Eventually, everyone figured out this was stupid, and Number Portability [1] was forced on carriers by regulation.<p>This problem is completely gone now. You can take your number with you.<p>If we allow people to take their data to new social networks, and force federation, then we will get decentralization. However, it won&#x27;t happen without regulation anymore than it did with the phone companies.<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Local_number_portability#History" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Local_number_portability#Histo...</a>

Beaker Browser is a cool experiment showing how you can decentralize the web that is both easy for end user&#x27;s and fun for developers because it pushes web standards.<p><a href="https:&#x2F;&#x2F;beakerbrowser.com" rel="nofollow">https:&#x2F;&#x2F;beakerbrowser.com</a><p>*disclaimer: I help develop Bunsen Browser, the mobile companion for Beaker Browser.

I&#x27;m not sure this is a coherent plan, since it doesn&#x27;t talk about how privacy rules get enforced for services. Who vets the services? If a fun game that you let have access to your &quot;personal data pod&quot; and it turns out to be Cambridge Analytica and just copies everything it sees into its own database, how is that an improvement over Facebook apps?<p>Choosing between service providers is no more meaningful for privacy than asking Windows users to download arbitrary apps. If smart phones are any more secure than desktops, it&#x27;s because Apple and Google are constantly improving OS-level security and policing their app stores for malware.<p>Of course app stores have well-known flaws. But if we want to do better than that, someone has to figure out a better way to choose good rules and enforce the rules better.

The people dont care about decentralization or centralization. This is all a big generalization, but humans are lazy and when it comes to making a moral choice, they&#x27;re going to pick the path of least resistance and completely ignore all moral consequences. Its looks like at the moment centralized services are what the people want and it&#x27;s what they deserve.

Shameless plug but I designed and wrote something for doing this from 2011 to 2015 because nothing like it existed or indeed exists as far as I&#x27;m aware.<p>It&#x27;s a p2p caching proxy that also lets you edit web pages collaboratively in realtime over a LAN or the internet. It has a contacts list system and p2p chat functionality. This project effectively died due to lack of interest and I still have various security concerns about it (Should you break&#x2F;reimplement Same-Origin policy or break&#x2F;reimplement the TLS chain of trust?)<p>The main security concern is that because it decentralises HTTP in-place (existing URLs can now be looked up on an arbitrary number of overlay networks if the original URL isn&#x27;t providing an OK response) it puts users at risk of malicious actors spamming overlay networks with browser exploits for popular resources like &quot;news.ycombinator.com&#x2F;&quot;.<p>I hope TBL and co converge on satisfying answers to these problems or constrain their design to not bother with decentralising existing URLs in-situ.<p>Code lives here: <a href="https:&#x2F;&#x2F;github.com&#x2F;Psybernetics&#x2F;Synchrony" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;Psybernetics&#x2F;Synchrony</a><p>Feel free to shoot me any questions.

Other people here have said this general idea here, the large centrealized services like Google and Facebook have succeeded in becoming so big through a lot of effort and a lot of cost, which was paid for by all the money they make. At a minimum they have to pay for their server use.<p>From what I understand the proposal here seems to not allow for the advertising model. I don&#x27;t think a services can grow and survive making people paying because people are too cheap.<p>There might be a better chance for something like this is they allow for the economics. - Maybe the data host can provide a &quot;advertising&quot; profile which the user has control of. This can be exposed to the application hosts to allow for advertising. - Maybe you also throw micropayments into the mix, along with bartering for information or micropayments.<p>Another issue is complexity. A number of comments have talked bout over-engineered solutions and protocols. This decentralizezd idea could be started with something small like an open social network standard. I think I saw something similar to this on HN not too long ago: - You have a web site, which is your profile. A provider could give you a nice editor for it. - You have a feed, where you can put pictures, short posts, long posts, whatever. This is distributed with RSS. (The host makes this all seamless for you.) - Identity is controlled with OAuth, used only to give an identity to visiting users. The owner users can manage permissions for certain remote users (his &quot;friends&quot;)<p>Such a service could be managed on your own web server, or there could be different cloud providers that make this arbitarily easy, with arbitrary levels of functionality on the &quot;profile&quot; page, the &quot;feed&quot; and the &quot;friend&quot; permission management.

&gt; From the above, it is clear that our primary obstacles are not technological [5]; hence Tim Berners-Lee’s call [6] to &quot;assemble the brightest minds from business, technology, government, civil society, the arts, and academia to tackle the threats to the Web’s future&quot;. Yet at the same time, computer scientists and engineers need to deliver the technological burden of proof that decentralized personal data networks can scale globally and that they can provide people with an experience similar to that of centralized platforms.<p>This whole article looks like &quot;well, the obstacles are not technological, but let me write a few pages about technology anyways&quot;.<p>If the obstacle are not technological, then we need non-technological solutions. So far I think GDPR is one such non-technological step towards taking back control of our personal data.<p>The hardest problem in my opinion is &quot;preventing the spread of misinformation&quot; because we essentially need a way to distinguish between malice and stupidity. Without mind-reading I do not see how this could be possible at scale.

I can&#x27;t escape the feeling that SOLID will be at best neutral but likely will make things worse. Some of the diagrams show <i>more</i> companies having access to your data where it will continue to be mined, sold, etc. If you control storage but not execution it seems like you control nothing.

The problem is people. To put it charitably, not everyone is &quot;technical&quot; enough to figure out how to own their own data, so I think silos and walled gardens are here to stay, because they are quick and easy for people. I for one, fully support keeping my own data in (as much as possible) future-proof formats, and although I&#x27;ve had a blog in some form for years, I want to move away from standard social media as much as possible.

The web cannot be decentralized without putting an end to SSL. As long as certificate-issuers are the arbiters of commerce and browsers push users to trust unsecured websites more and more, malicious governments will be able to silence people by revoking their certs.<p>There are stronger alternatives. We need to make a push to begin using them.

I went to the Decentralized Web Conferece a few years ago and really liked it. In spirit, I am onboard.<p>In practice, I am satisfied with just using my own domain for email, my web site, and self-hosted blog. For communication I like FaceTime so I can see people while I am talking with them, phone, and email.<p>I still use social media, very occasionally, to see what people are doing and sometimes advertise my new open source projects and updates, and any books I write. Most of the problems people talk about with Facebook&#x2F;Twitter don’t bother me as long as I only use the systems infrequently. I am not tempted to cancel my accounts.

The design, typography, and diagrams in this article are wonderful. I like it when people pay this much attention to detail!

This essay is in radical need of a TL;DR. If something is this important, you owe it to the subject matter not to bury the lede under a mountain of history and flowery exposition.<p>Ask yourself: who is this for? People who are not already deeply passionate will stop reading unless they are engaged in a minute of reading. Note that a minute is being extremely generous; on a commercial consumer site, it&#x27;s apparently an average of 7 seconds before someone will click away.<p>I recommend that you check out this video and reconsider how you might reframe your message as a call to action that speaks to a better future we can create together.<p><a href="https:&#x2F;&#x2F;youtu.be&#x2F;qp0HIF3SfI4?t=121" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;qp0HIF3SfI4?t=121</a><p>I even jumped you to the good part.

I’m all for the principles here, but one worry I have is the loss of efficiencies afforded by economies of scale which could dramatically increase the carbon footprint compared to the centralized versions.

Decentralized web can be downloaded and backed up by one entity. Then, you can go to that centralized entity to enjoy all the content.<p>If we still don&#x27;t have decentralization, it&#x27;s because it is not as easy.

In 2005, I worked at a startup that attempted to solve the problem of privacy and security for personal information (photos, home videos, music, personal health&#x2F;finance documents, contacts etc) while also providing ability to share and collaborate.<p>The solution involved running a mesh network with nodes on user&#x27;s laptop or desktop and a corresponding node in the cloud. These nodes would index local data and provide replication of metadata across nodes and backup of actual data to cloud node.<p>A locally running web app acted as replacement for &#x27;windows explorer&#x27;. It allowed the user to access all their files and folders across all their nodes, access them (open document, play music&#x2F;videos, see contacts etc), create smart collections and share these files, folders or collections with other users in a secure authenticated and private manner.<p>User got an identity - which comprised of a dedicated domain (or subdomain) and a PKI certificate tied to that domain. Each node had it&#x27;s own private key and their public keys were tied together by the identity certificate.<p>All communication between nodes (of same user or across users) where authenticated and encrypted using these identity&#x2F;node keys and certificates. No central node existed in the system that could spy on these activities. The architecture separated the network discovery cloud nodes from your data cloud nodes and architecture allowed for your data cloud nodes to be hosted separately anywhere (say, in your own cloud instances).<p>This is the only system I have seen that utilized zero knowledge protocols and made it accessible to common people to manage their data and share with others as well.<p>But unfortunately, as a business it never took off. It got acquired by emc and merged with mozy (good old data backup company) and then this product died a silent death in 2010.<p>Maybe it was timing, maybe after snowden, if this product had launched it would have done well.<p>But now, I think a more urgent and a relatively less complex problem to solve is one of distributed communication. In this era of always connected powerful devices (mobile phones, home gateways), why don&#x27;t we all have our own personal email&#x2F;chat servers that nobody else can spy on? Why does email and chat have to get relayed via big aggregators who mine so much data as well as metadata?<p>Not only do they violate privacy, they succumb to security breaches and cause serious damages.<p>I feel the stage is set for this disruption: crypto protocols, always-on cheap connectivity, compute power at the edge, and sensitivity to privacy&#x2F;security in general population – all of these ingredients are appropriately set right now for this to happen.

The point about the decentralised web allowing the permissionless creation of centralising systems reminds me of the paradox of tolerance, where tolerant societies are thought to be taken over by intolerance if they tolerate that intolerance.<p>Maybe this is a lesson that we need to be less tolerant towards the creation of centralised services because those with money and power will seek to bring decentralised systems under their own control.

For the technically savvy, you can run a virtualized desktop:<p><pre><code> - GPU passthrough VM (gaming) - SATA passthrough (FreeNAS) - multi NIC passthrough (pfSense&#x2F;OpenWRT) - app server&#x2F;cloud&#x2F;P2P Linux or FreeBSD VM(s) </code></pre> <a href="http:&#x2F;&#x2F;unraid.net" rel="nofollow">http:&#x2F;&#x2F;unraid.net</a> sells a KVM-based product. VMware ESXi and XenServer are free. Connect a Ubiquiti AC-Lite WiFi access point to a dedicated NIC on the x86 box, WAN to another NIC. Since pfSense owns the WAN NIC, it can host a VPN server for your devices, including mobile. All VMs get virtual NICs. Dell T30 with quad-core Xeon and ECC costs about $400 with 8GB RAM and 1TB disk, it can hold 4 x 3.5&quot; drives (20 TB in RAID-1) and 2 x 2.5&quot; SSD.<p>Level1Techs has intro videos on home servers: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;results?search_query=level1+home+server" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;results?search_query=level1+home+ser...</a><p>Advantages:<p><pre><code> - Stable and boring x86 platform - Good performance for gaming - Commercially supported hardware - Upgradeable storage and GPU - Upgradeable router software</code></pre>

Great article Ruben! I&#x27;ve been following Solid&#x27;s progress for a while, and I think your article very eloquently summarize its purpose and relevance. I&#x27;m especially interested in the ability to circumvent the middle-men, and resolve the marketplace chicken-and-egg problem once and for all.<p>Watching your TED talk in 2013 was one of the most influential moment in my life, and discovering the semantic web was perhaps my greatest epiphany. While the vision never left my mind, I never acted on it. Until now.<p>I&#x27;m dedicating 2019 to linked data. I&#x27;m going all-in.<p>Last week, I started to build a tool to convert unstructured input to linked data. Even after recognizing canonical literals (email, phone, url, color, gender, boolean, integer, float, date, time span, money, weight, distance, language, image, geo coordinates), I couldn&#x27;t accurately infer predicates and guess classes. Before trying more complicated stuff like bayesian inference, I decided to try a simpler exercise.<p>This time, I want to aggregate structured data from different sources and map it to some existing ontologies. For example, I want to convert some JSON about comments and links from Reddit and Hacker News to RDF using the <a href="http:&#x2F;&#x2F;schema.org" rel="nofollow">http:&#x2F;&#x2F;schema.org</a> vocabulary.<p>- Can I feed the JSON into some ML system that automatically figures out the mapping? What if I provide some annotation or feedback?<p>- Can I manually turn the JSON into JSON-LD and use that as the mapping information? What about complex transformations (different structures and literals)?<p>- Should I implement the mapping manually using my favorite programming language?<p>- Should I use R2RML or RML?<p>What&#x27;s the state of the art today for semantic data integration?

Solid looks to be trying to reimplement what platforms like Ethereum are already building. The same ethos is there and this is very well written but I wonder if the Solid project just missed that when doing their research. Hopefully all of their efforts don&#x27;t go to waste and they can extend some of their work to the broader decentralized web community.

&lt;i&gt; He and many others were able to state their critical opinions because they had the Web as an open platform, so they did not depend on anyone’s permission to publish their words. Crucially, the Web’s hyperlinking mechanism lets blogs point to each other, again without requiring any form of permission. This allows for a decentralized value network between equals, where readers remain in active and conscious control of their next move.&lt;&#x2F;i&gt;<p>For decentralization the root problem always existed, while pointing at another resource requires no permission, receiving and hosting that resource does. Your government has to let you receive it and your ISP has to let you host.<p>This is a much lower level problem compared to the three challenges Berners-Lee puts forward, which seem to have little to do with decentralization.<p>1. taking back control of our personal data;<p>2. preventing the spread of misinformation;<p>3. realizing transparency for political advertising.

I think if you add some cryptography to Solid and use JSON-LD and pick some schemas and not expect everyone to implement OWL and then get a usable naming scheme for IPFS (or replace IPFS with something similar with names that work) and then create some P2P Solid servers then this could work pretty well.

What&#x27;s to stop a service from having a pod that stores user&#x27;s data in a mutated form? (Forgive me for the basic question).

Somebody should make an easy to install home server with a standard API to access data. People will start decentralizing on their own when you can just buy a box and do some basic configuration, and have a secure home web server. And then developers will build on the decentralized platform because it has users.

&gt; Since 2010, no single browser has gained more than two thirds of global market share anymore<p>What about Google Chrome?

Maybe I missed it somewhere, but I didn&#x27;t quite follow how this is going to gain a foothold. Solid has the same problem any new social media platform has - before people want to use it, people have to be using it. Facebook and Google certainly have no incentive to promote it.

The specification (set of protocols) is here <a href="https:&#x2F;&#x2F;github.com&#x2F;solid&#x2F;solid-spec" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;solid&#x2F;solid-spec</a>

Missed that about 20% humanity is under a walled national garden. If you have a protocol that are individual or home oriented, would it be allowed to work even.

<i>Since 2010, no single browser has gained more than two thirds of global market share anymore</i><p>Pretty sure Chrome did. Or WebKit&#x2F;Blink family. This is GOOD imho.

I don&#x27;t see any real possibility for decentralization so long as HTTP(S) is the protocol of the web.

This sounds like yet another technical solution to a problem that is mostly societal.

All of this is very academic.<p>Regular people and businesses are always going to make the decision in front of them.<p>&#x27;Decentralization&#x27; unto it&#x27;s own, is not something anyone directly cares about. People care about privacy, somewhat, but there are other paths to privacy, or at least, consumers may very well believe there are.<p>Decentralization will only happen with a real impetus: a product or service that facilitates it, that people want, either for issues related to decentralization, or, more likely for some other reason that just happens to facilitate decentralization for some other, related reason.

This is a load of ... You can&#x27;t descentralize the web for 2 reasons: DNS and SSL. And then you have the IP organisation, the name escapes me right now.

Huh...<p>&gt; The situation becomes problematic when we are robbed of our choice, deceived into thinking there is only one access gate to a space that, in reality, we collectively own.<p>Robbery - the action of taking property unlawfully from a person or place by force or threat of force. [0]<p>Deceit - The action or practice of deceiving someone by concealing or misrepresenting the truth [1]<p>That&#x27;s what those words mean. They also have nothing to do with anything that has happened with the internet over the last 20 years.<p>[0] <a href="https:&#x2F;&#x2F;en.oxforddictionaries.com&#x2F;definition&#x2F;robbery" rel="nofollow">https:&#x2F;&#x2F;en.oxforddictionaries.com&#x2F;definition&#x2F;robbery</a><p>[1] <a href="https:&#x2F;&#x2F;en.oxforddictionaries.com&#x2F;definition&#x2F;deceit" rel="nofollow">https:&#x2F;&#x2F;en.oxforddictionaries.com&#x2F;definition&#x2F;deceit</a>