Hi HN! Matrix-CRDT connects the worlds of Yjs [1] (a proven, high performance CRDT) with that of Matrix.org [2].<p>It started as an experiment, asking myself; can we store "state updates of a datastore" in Matrix instead of chat messages? Now, I'm convinced it's actually a really powerful combination to develop real-time, collaborative software.<p>I'm using it for a new project and so far didn't have to write a backend yet. Matrix takes care of a lot of stuff: Authentication, E2EE, federation, hosting, etc. - so I can focus on the client.<p>I love the ideas of Local First [3] software. Personally, I'm convinced the technologies in this ecosystem (CRDTs, etc) are really powerful and can do more to decentralize software than many web3 technologies that currently receive much of the hype.<p>However, it's still early days, so I decided to open source parts that I think will be useful to others (I recently also shared SyncedStore [4])<p>[1] https://github.com/yjs/yjs<p>[2] https://www.matrix.org<p>[3] https://www.inkandswitch.com/local-first/<p>[4] https://www.syncedstore.org
See, this is what I keep going on about to my friends - Matrix is actually a graph syncing protocol, not an instant messaging protocol, and is a great way to build federated applications.<p>I get that Element needs to focus on one use case in order to pay the bills, but there's a lot of room for applications that have barely been explored - calendars, contacts, games, social media, heck - you could build a federated gitlab-like collaboration site with truly federated data. You get multi-device sync, e2e encryption, and identity management out of the box.<p>Thanks for putting this together, I hope there will be a gold rush as people realize how much low-hanging fruit there is!
> Personally, I'm convinced the technologies in this ecosystem (CRDTs, etc) are really powerful and can do more to decentralize software than many web3 technologies that currently receive much of the hype.<p>100% agree! Web3 is really only tackling monetary instruments, which is interesting but very narrow. The majority of apps people need don't require global consensus, and it's too slow and expensive anyway. Yet, there's so much interesting stuff that can be decentralized/federated.<p>> Matrix takes care of a lot of stuff: Authentication, E2EE, federation, hosting, etc. - so I can focus on the client.<p>This is great to hear! What else besides CRDTs do you think is needed in order for matrix to be a platform for general purpose apps like this?
This is an interesting use of the Matrix protocol. I like the many creative ways Matrix is being used outside the chat space, where its predominant use lies. However, I have several questions about this approach.<p>- What happens with E2EE when there are key exchange problems? (i.e. a server goes down and the account is moved)<p>- What happens when someone redacts a message? I assume you'd want to disable redactions for these rooms or you'll end up with broken update trees!<p>- How do you communicate clearly that any data you add is in the event chain forever? In Word and GDocs you can remove old revisions of a document, but I don't think you can in this system? That could be a feature, of course!<p>- What's the performance impact of such a system on a server editing reasonably large documents? If twenty people each edit ten to twenty documents you end up with quite a large set of state, especially over time!
Awesome project!<p>I have said it before and I will say it again, I think this year is going to be the year of Yjs. The ecosystem around it is brilliant and I believe it will become the leading CRDT framework (it probably already is).<p>Happy to see you using TipTap too for the demo, a brilliant rich text editor framework (with support for Vue.js, React and Svelte) built on top ProseMirror with first class Yjs support.<p><a href="https://tiptap.dev" rel="nofollow">https://tiptap.dev</a><p>The TipTap guys (Philipp and Hans) along with Kevin Jahns of yJS have stated the y-collective inorder to centralise funding and organising of the Yjs ecosystem:<p><a href="https://opencollective.com/y-collective" rel="nofollow">https://opencollective.com/y-collective</a><p>Personally I'm excited about the us of Yjs and CRDTs outside of just rich text editing, I think there is great potential to build a distributed offline enabled database targeting mobile and PWAs with it. Something like PouchDB but with automatic conflict resolution.<p>Finally there is a Rust port in development to improve the (already very good) performance and make it cross platform with other languages.<p><a href="https://github.com/y-crdt/y-crdt" rel="nofollow">https://github.com/y-crdt/y-crdt</a>
Amazing work. I think there's a whole host of apps that just store some small amount of data for each user that could work entirely on something like this.<p>Some nice things:<p>- Anyone can host their own Matrix homeserver, which can then sync its data within a federation, allowing users pretty fine-grained control over what data they share and with whom.<p>- Add end-to-end encryption, and it becomes even better: data stored somewhere in the cloud (i.e. the user doesn't have to manage a server), but the server knows almost nothing about what you've stored aside from some metadata.
Thank you for your work! First SyncedStore, now this. Yjs keeps getting better and is already my favourite technology to work with. Going to include this in my app :) <a href="https://app.lity.cc" rel="nofollow">https://app.lity.cc</a>
I recall that matrix itself has some CRDT-like data structure to transmit messages. If that's accurate, could you map some of the functionality of yjs directly into native matrix ops?
(dreaming big here) You could make this a basis for a generic conversational UI[1] for web data. It wouldn't be a perfect UI for most applications, but it provide a couple of important features:<p>1. It would be easy to manage data sharing. Post updates to data in channels that the relevant people have access to.<p>2. It would be easy to debug data changes. I could see it becoming the equivalent of a terminal app but for the web.<p>[1] <a href="https://en.wikipedia.org/wiki/Conversational_user_interface" rel="nofollow">https://en.wikipedia.org/wiki/Conversational_user_interface</a>
I've used yjs for a project that aims at computing individual climate footprints as a group in a conference.<p>It works beautifully with almost no code. I used Yjs P2P, but P2P is blocked in numerous networks.<p>If you want to try, with an hackernews room, but sorry it's in french : <a href="https://nosgestesclimat.fr/conférence/hackernews" rel="nofollow">https://nosgestesclimat.fr/conférence/hackernews</a><p>So using Matrix as a backend is a great idea.<p>The only caveat I found to Yjs is that if you want to persist data (and not lose it when the server crashes), nothing's really plug-and-play, so I went with Supabase.
Yjs looks interesting; I'm currently building a management UI that for now is a very straightforward set of REST APIs and a database, but there's a lot of concerns about people editing the same configuration simultaneously; I wonder if something like yjs can be used to create a collaborative editing environment.<p>I'd probably have to re-architect the whole front- and back-end though. Although on the other hand, there will be some code editors (currently using Monaco) where this could be usable.
Having an Datomic like store backed by something like this.<p><a href="https://github.com/replikativ/datahike" rel="nofollow">https://github.com/replikativ/datahike</a><p>Is an Open Source variant of Datomic.<p>Lambdaforge wants to eventually have this work with CRDTs.<p>Using the Matrix ecosystem for this is quite interesting as it solves many problems for you already.
Lots of Matrix ignorance here - so sorry if this is a dumb question.<p>When you auth with Matrix, is that a centralized action (one company / org behind it) or is that a decentralized action (its more a protocol for auth amongst the community of users)?<p>ie. Is using Matrix for auth decentralized or a different centralization?
Just to understand, is the concept here that you send all of the updates for a particular data item to a Matrix "room" and then converge on the current state of that data item by each client replaying that data?
This sounds like (at least a big step towards) distributed event sourcing which is something that I have been anticipating for a while now but haven't heard any news of.
Is there any community where projects like this have a center of gravity? (eg. conferences, Discord, Matrix, ...) for people interested in getting involved in the community?