Why Has Figma Reinvented the Wheel with PostgreSQL?

I&#x27;m at a company that is weighing a very similar decision (we are on RDS Postgres with a rapidly growing database that will require some horizontal partitioning). There really isn&#x27;t an easy solution. We spoke to people who have done sharding in-house (Figma, Robinhood) as well as others who migrated to natively distributed systems like Cockroach (Doordash).<p>If you decide to move off of RDS but stay on Postgres, you can run your own Postgres but now lose all the benefits of a managed service. You could move off of AWS (eg to Azure), but moving to a different cloud is a huge lift. That, btw, would also be required if you want to try something like Spanner (move to GCP). Moving off of Postgres to another database is also risky. The migration will obviously take some effort, but you&#x27;re also probably talking about lots of code changes, schema changes, etc, as well as unknown operational risks that you&#x27;ll discover on the new system. This applies if you&#x27;re talking about things like Cockroach or even moving to MySQL.<p>That said, rolling your own sharding is a MASSIVE undertaking. Limitless looks promising, since it takes care of a lot of the issues that Figma ended up spending time on (basically, you shouldn&#x27;t need something like Figma&#x27;s DBProxy, as shard routing, shard splitting, etc will be taken care of). It&#x27;s still in preview though, and like the article mentioned, the costs may be high.<p>Overall, no easy decisions on this one, unfortunately.

I’m definitely of the opinion that what Figma[0] (and earlier, Notion[1]) did is what I’d call “actual engineering”.<p>Both of these companies are <i>very</i> specific about their circumstances and requirements<p>- Time is ticking, and downtime is guaranteed if they don’t do anything<p>- They are not interested in giving up the massive amount feature AWS supports via RDS, very specially around data recovery (anyone involved with Business Continuity planning would understand the importance of this)<p>- They need to be able to do it safely, incrementally, and without closing the door on reverting their implementation&#x2F;rollout<p>- The impact on Developers should be minimal<p>“Engineering” at its core is about navigating the solution space given your requirements, and they did it well!<p>Both Figma and Notion meticulously focused on the minimal feature set they needed, in the most important order, to prevent disastrous downtime (e.g Figma didn’t need to support <i>all</i> of SQL for sharding, just their most used subset).<p>Both companies call out (rightfully so) that they have extensive experience operating RDS at this point, and that existing solutions either didn’t give them the control they needed (Notion) or required a significant rewrite or their data structures (Figma), which was not acceptable.<p>I think many people also completely underestimate how important operational experience with your solution is at this scale. Switch to Citus&#x2F;Vitess? You’ll now find out the hard way all the “gotchas” of running those solutions at scale, and it would guarantedly have resulted in significant downtime as they procured this knowledge.<p>They’d also have to spend a ton of time catching up to RDS features they were suddenly lacking, which I would wager would take much more time than the time it took implementing their solutions.<p>Great job to both teams!<p>[0] <a href="https:&#x2F;&#x2F;www.figma.com&#x2F;blog&#x2F;how-figmas-databases-team-lived-to-tell-the-scale&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.figma.com&#x2F;blog&#x2F;how-figmas-databases-team-lived-t...</a><p>[1] <a href="https:&#x2F;&#x2F;www.notion.so&#x2F;blog&#x2F;sharding-postgres-at-notion" rel="nofollow">https:&#x2F;&#x2F;www.notion.so&#x2F;blog&#x2F;sharding-postgres-at-notion</a>

Good points. Although, having worked on many high scale architectures before, I always err on the side of thinking that any technical solution of this magnitude would have far too many nuances for a blog post to capture. And I believe Figma’s post also talks mainly about the common denominator that’s easier to communicate to the external world.<p>For me to understand their choices, I’ll first have to understand their system, which can be a task in itself. Without that, I’d not become aware of the nuances, only general patterns.

Aren&#x27;t there any good managed postgres solutions supporting citus? The decision here seems to have been to invent a whole new sharding solution instead of building enough in house DBA to self-host postgres (if you want to stay on Amazon, you can use any extension you want on EC2). Speaks for the state of engineering right now.

The biggest issue with the Figma article was they did not discuss partitioning the tables before they sharded them.

3rd party solutions can also add complexity you don&#x27;t want. You need to keep up-to-date with their release schedules to have access to bug and security fixes, even though you would feature wise be happy with the older version.<p>Also these can add unnecessary complexity by having features you don&#x27;t need. Or they might be missing features you need. Contributing up-stream can be difficult and there might be conflicts of interest especially for projects which have separate paid version.

Nice article, I also wondered why they omitted citus. Is there any plan from rds to offer it?<p>Obviously it could interfere with demand for aurora limitless

Now seems a good time to point out, the wheel has literally been reinvented over and over again. The wheels of yesterday were terrible. Each version gets better. It&#x27;s fine, reinvent away folks :)

I still can&#x27;t understand why they decided to use a single database for all their customers. If each customer needs access to its own data, why not a dedicated database for every customer?

Even notion has a similar approach to sharding postgres but both of them could benefit from having shard IDs prefixed with YY&#x2F;MM&#x2F;DD(as needed) otherwise it&#x27;s back to the shard navigator once they max out against org-ids for each shard&#x27;s capacity

The answer is obvious: they invented their own sharding solution because it&#x27;s a really really cool problem to work on and they have more engineers than they really need to develop their actual product. A more resource-constrained team would have found a solution that sharded their backend using one of the existing solutions out there.<p>I have seen this several times before and it&#x27;s always a symptom of having too many engineers working below the waterline. Rather than work on the actual customer-facing problem, let&#x27;s port the backend to do event-sourcing&#x2F;cqrs, move all our infrastructure to k8s, change language from x to y etc.<p>These are all what I would call &quot;internal goals&quot; (ie they may or may not be necessary or even essential to progress but are not directly customer-visible in their outcomes even if they may enable customer features to be built or indirectly improve the customer experience later) and need to be held to an extremely high level of scrutiny.<p>If you&#x27;re amazon&#x2F;google&#x2F;meta and you need to do this because of extreme user scale I might believe you. If you&#x27;re CERN or someone and you need to do this because of absolutely ridiculous data scale I might believe you. The idea that it&#x27;s better for figma to write their own sharding solution than it is to port to one of the existing ones just doesn&#x27;t pass even the most basic sniff test.

The article should be titled &quot;Why Figma HAD TO reinvent the wheel with PostgresSQL&quot;. When you have a legacy system and not enough time, or will to move off of it the only option is to get inventive and build with what you have.<p>There is always a price. In this case the database team did something quick, cheap and easily. But the Application teams now have to deal with handling all the nuaces of the system. Maybe Figma has more people in these Apps teams with time on their hands to handle it.

If you go with sharding proxy design why not use Apache ShardingSphere?<p>It follows the same approach but is far more sophisticated and mature.

i wonder how neon would perform with a database of this size. Several terabytes per table is pretty big.

I find the wheel is most typically reinvented in pursuit of venture capital.

Am I the only one here thinking they should have just used MongoDB and be done with it?<p>I know over simplified approach but majority of problem would be solved.