> The tricky part is that when multiple users open the same Fig file concurrently, Figma’s infrastructure needs to ensure that they are all connected to the same server. That server can then be the sole authority on the state of that document, and write to it without race conditions.<p>This is the killer app for CloudFlare's new Durable Objects. They solve the routing and data distribution layer, allowing you to write single-threaded business logic to coordinate changes.<p>They even have a transactional storage capability which is priced more or less equivalent to DynamoDB (which Figma uses for their write-ahead log).<p>This pattern also helps scale write transactions without melting your database. <a href="http://ithare.com/scaling-stateful-objects/" rel="nofollow noreferrer">http://ithare.com/scaling-stateful-objects/</a><p>I try to boost Durable Objects every chance I get, in order to "push the battleship" in the direction of another cloud provider implementing something equivalent.<p>While this article is written by the plane.dev team, which has an adjacent product, their approach seems more geared towards more demanding backends. Lots of use cases don't need to run e.g. a game or simulation on the backend, they just need a synchronization layer in front of a document store.<p>---<p>> They could stuff the whole document into a binary blob or json(b) column next to the metadata.<p>In my experience doing this in MySQL... do not do this. Once you have a dozens-if-not-hundreds-of-gigabytes large table full of JSON, it becomes a real nuisance. As a halfway measure, I think it would help to have a "blobs only" table separate to the metadata table. But as the OP points out, it is not economical anyway.
Although the tech behind Figma is nothing but impressive I find that the biggest downside to current collaborative tools is that they're expected to be used collaboratively <i>at all times</i>.<p>I would say that no more than 10% of the time I spend on Figma is for collaborating with other people in short brainstorming sessions, team workshops, etc. The other 90% is spent by myself working and polishing said prototypes, but still having to deal with the loading times, server hiccups, and so on.<p>An "offline mode" of sorts would also be a silly feature to expect since their entire stack is built around collaboration. Seems like a difficult balance.
Alternately: file formats and saving to disk are just a very primitive database where the cache is the size of the entire database (you load the entire thing into memory). Also you don't have good disaster recovery (see "I forgot to save and lost the last half hour of work!"). Also, auto-save is a very primitive form of a write ahead log
I'm more impressed by the architecture, which resembles that of room-based online games. After all, online game is the ultimate real-time collaboration system. The only difference is that, in figma, room states are saved (backed by S3 and the lousy "disk" cache).<p>The file <-> database contrast feels rather moot, because it's natural to have metadata in usual DBMS and blobs in object/file storage (or in blob column).
One note that the article makes is that reading from S3 requires a full read of the file. As an alternate approach is to use HTTP range headers to read only part of a file and minimise how much data goes over the wire. I'm not sure if the 'kiwi' file format would support that (and this is only for reads I believe).<p>A nice demo of this for sqlite is here - <a href="https://phiresky.github.io/blog/2021/hosting-sqlite-databases-on-github-pages/" rel="nofollow noreferrer">https://phiresky.github.io/blog/2021/hosting-sqlite-database...</a>
Thanks for taking time to reverse engineer how Figma works on background. Wondering if info like saving buffer changes to DynamoDB are published by Figma on a blog or you just inspected this behavior in browser, the same for S3 and postgress?<p>After all, Figma is an excellent product except the loading time on existing documents 5mb or more, design files, take time to open up reminding me Photoshop 7.0 days.
I think I stumble into the trap of being like “ok single-master OT with a warm failover and probably an append only log to EBS or something, consistency as logical monotonicity yada yada” because I’m old and jaded.<p>But this creates at least two problems right off the top of my head:<p>1. When I first got OT working at a coffee shop in SOMA I said “hell yeah” so loud everyone looked at me. It’s magical stuff and it’s way more fun to be an active part of others finding all this amazing shit we’ve inherited.<p>2. There are probably a zillion refinements since I last did any of that stuff and by glossing over it with a mental yawn, I’m probably actually falling behind the cutting edge in ways I definitionally don’t see.<p>Im going to try to pay more attention to the details of stuff I <i>think</i> I already know.