I'm looking for peer to peer framework recommendations to build a distributed app. In particular, a framework that supports seeds and peer discovery, message passing, maybe firewall / UPNP support, etc.<p>I'm not particular about language (Java, Go, Javascript, etc), since it's for a new project. I found this thread from 2016 (https://news.ycombinator.com/item?id=11145691), it had some interesting recommendations, but quite a few were out of date, and with the big push to distributed apps nowadays, I thought it might be a good time to readdress the topic.<p>Some of the solutions form the other thread or that I've found:<p>- Smokesignal (Javascript / Node, not updated in >10 months) https://github.com/marcelklehr/smokesignal<p>- ZeroTier: https://www.zerotier.com/product-netcon.shtml<p>- Maki: https://maki.io<p>- Telehash: http://telehash.org/<p>- A whole host of blockchain projects, Bitcoin, Ethereum, HyperLedger, Scorex (Scala blockchain)<p>Bitcoin is an interesting option, it certainly has a large user base and is well tested and secure code, but forking it has it's own set of issues.
Maybe Scuttlebutt? <a href="https://www.scuttlebutt.nz/" rel="nofollow">https://www.scuttlebutt.nz/</a><p>Scuttlebutt is advertised as a social network, but in core it is an eternal distributed log. You can pass either public or private messages and assets around.
Awesome List of P2P resources:<p><a href="https://github.com/kgryte/awesome-peer-to-peer" rel="nofollow">https://github.com/kgryte/awesome-peer-to-peer</a><p>Personal bias tilts to web tech. WebRTC will require you to centrally implement all your peer broker and logic. But take a look at something like Peer5 Video CDN to appreciate scalability:<p><a href="https://www.peer5.com/" rel="nofollow">https://www.peer5.com/</a>
<a href="https://urbit.org/docs/" rel="nofollow">https://urbit.org/docs/</a><p>The idea with urbit (amongst others, it's a big system) is for everyone to have a personal server that's easier to administrate than a linux server. It's been posted a bunch of times <a href="https://hn.algolia.com/?query=urbit&sort=byPopularity&prefix&page=0&dateRange=all&type=all" rel="nofollow">https://hn.algolia.com/?query=urbit&sort=byPopularity&prefix...</a><p>I've also heard about <a href="https://zeronet.io/" rel="nofollow">https://zeronet.io/</a> and <a href="https://beakerbrowser.com/" rel="nofollow">https://beakerbrowser.com/</a>
I’m curious what your goal is. Do you want the complete immutability of transactions like bitcoin/blockchain; or do you want the peer/seed capability like BitTorrent and IPFS with a DHT (Kademlia)? I’d be interested in seeing a good list of the technology choices that can make up a P2P project if anyone knows of one, please share.<p>But you definitely should checkout IPFS and Kademlia.
Maybe check out Dat[0] - especially useful if you need some kind of distributed data store. The main implementation is in Javascript.<p>[0] <a href="https://datproject.org/" rel="nofollow">https://datproject.org/</a>
The 181 pages long GNUnet dissertation very recently got released:
<a href="https://grothoff.org/christian/habil.pdf" rel="nofollow">https://grothoff.org/christian/habil.pdf</a><p>I'd strongly recommend going for GNUnet, nobody takes me serious when I say this, but it WILL surplant IP eventually.
The WebTorrent Project [1] has a whole suite of tools for decentralized applications based around BitTorrent. It's not a framework per-se, but it breaks up the protocol into libraries for your use.<p>If you're interested in Ethereum, it has other non-blockchain technologies that you can use for your apps. It has the Swarm network for storing bits of data, the Whisper network for chat and messaging, as well as Public-Key signing to verify messages. I'm using the latter functionality for a demo to replace password as the security mechanism.<p>[1]: <a href="https://github.com/webtorrent" rel="nofollow">https://github.com/webtorrent</a>
Secure Scuttlebutt. It's probably the only one seeing everyday use to get things done. Patchwork, the flagship app is as good as Facebook/Twitter. Scuttlers often use git over SSB, even.<p>What SSB gets right and the others don't, is the incentive to mirror or replicate. Virtual currency is a weak incentive in an early stage network - but friendship isn't; you mirror your friends' data.<p>The basic API to program the dapp can be seen here - <a href="http://scuttlebot.io/" rel="nofollow">http://scuttlebot.io/</a> But for more advanced uses see <a href="https://youtu.be/f_baWUW4R8Y" rel="nofollow">https://youtu.be/f_baWUW4R8Y</a>
Maybe checkout the dat project <a href="https://datproject.org/" rel="nofollow">https://datproject.org/</a> which is whats used by BeakerBroswer etc.
Another project you might find useful, that we at <a href="https://www.maidsafe.net" rel="nofollow">https://www.maidsafe.net</a> are actively working on, is <a href="https://github.com/maidsafe/crust/" rel="nofollow">https://github.com/maidsafe/crust/</a>.<p>It's a generic peer-to-peer networking library written in Rust.<p><pre><code> * it connects two peers together
* various NAT traversal techniques are implemented: hole punching, UPnP/IGD, etc.
* it supports both TCP and UDP. In case of UDP it uses uTP: http://www.bittorrent.org/beps/bep_0029.html
* all messages are encrypted
* it implements automatic peer discovery on LAN
* it caches it's previous connections and is able to reuse them in the future
* etc.</code></pre>
The virtual synchrony protocols don't get enough love. Conceptually, they're sort of like if you performed PAXOS to determine ordering for a distributed finite state machine. Let's you build all sorts of distributed applications where you trust the nodes.<p><a href="https://en.wikipedia.org/wiki/Virtual_synchrony" rel="nofollow">https://en.wikipedia.org/wiki/Virtual_synchrony</a><p><a href="http://www.jgroups.org/" rel="nofollow">http://www.jgroups.org/</a><p><a href="https://archive.codeplex.com/?p=vsync" rel="nofollow">https://archive.codeplex.com/?p=vsync</a>
Re blockchain technologies: while there is an obvious intersection between the world of blockchain and P2P, blockchain technologies are slow because BFT adds many requirements to the game. Basically blockchain technologies use P2P to reach a common state and this could not be necessary in your use case.<p>You can take a look at OpenBazaar and Particl for P2P implementations.
ZeroMQ <a href="http://zguide.zeromq.org/py:all" rel="nofollow">http://zguide.zeromq.org/py:all</a> ... "In One Hundred Words: ZeroMQ (also known as ØMQ, 0MQ, or zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fan-out, pub-sub, task distribution, and request-reply. It's fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multicore applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. ZeroMQ is from iMatix and is LGPLv3 open source."
Peer Name Resolution Protocol (PNRP)<p>- Developed by Microsoft<p>- Open Specification<p>- DHT-based<p>- IPv6 only<p>- Windows support since Windows XP SP 3<p>- .NET support since .NET Framework 3.5 (System.Net.PeerToPeer)<p>[1]: <a href="https://en.wikipedia.org/wiki/Peer_Name_Resolution_Protocol" rel="nofollow">https://en.wikipedia.org/wiki/Peer_Name_Resolution_Protocol</a><p>[2]: <a href="https://msdn.microsoft.com/en-us/library/aa371699.aspx" rel="nofollow">https://msdn.microsoft.com/en-us/library/aa371699.aspx</a> "PNRP Namespace Provider API"<p>[3]: <a href="https://msdn.microsoft.com/en-us/library/bb726971.aspx" rel="nofollow">https://msdn.microsoft.com/en-us/library/bb726971.aspx</a> "Peer Name Resolution Protocol"
Blockstack looks like it would fit here:<p><a href="https://github.com/blockstack/blockstack/blob/master/README.md" rel="nofollow">https://github.com/blockstack/blockstack/blob/master/README....</a>
If you plan on having any collaborative editing features, Yjs [1] is a p2p framework utilizing CRDTs with support for IPFS.<p>[1] <a href="https://github.com/y-js/yjs" rel="nofollow">https://github.com/y-js/yjs</a>
I recently discovered zyre. Its from the same guy that brought us the wonderful zeromq library<p>While it does a lot of the connection and communication part, it lacks a way to bypass nat routers.<p>Here is a book on zyre <a href="http://zguide.zeromq.org/php:chapter8" rel="nofollow">http://zguide.zeromq.org/php:chapter8</a><p>I also found this thing called pwnat which bypasses nat routers without the need for UDP hole punching !<p><a href="https://github.com/samyk/pwnat" rel="nofollow">https://github.com/samyk/pwnat</a><p>I feel like if someone made a high level wrapper combining these two, we can have a very good framework.
> supports seeds and peer discovery, message passing, maybe firewall / UPNP support, etc.<p>Our audacious 'IPv8' work is now deployed live. Might fit you needs. Includes above features in a low-complexity approach.
<a href="https://github.com/Tribler/py-ipv8" rel="nofollow">https://github.com/Tribler/py-ipv8</a>
Our fresh IETF Internet Standard draft of this work:
<a href="https://tools.ietf.org/html/draft-pouwelse-trustchain-00" rel="nofollow">https://tools.ietf.org/html/draft-pouwelse-trustchain-00</a>
TomP2P (<a href="https://tomp2p.net/" rel="nofollow">https://tomp2p.net/</a>) is a library handling firewall / UPNP support, however, the focus is academic/experimental.
Worth having a look into the Holochain framework:<p><a href="https://holochain.org/" rel="nofollow">https://holochain.org/</a><p>Depending on the application(s) you have in mind.
Maki 0.3 will include components from Fabric: <a href="https://fabric.fm" rel="nofollow">https://fabric.fm</a> — automatically available to anyone upgrading from 0.2. :)