At my startup Tamber (https://tamber.com) we have had to deliver very high database I/O without breaking the bank. Our solution provides two main features:<p>1. Database workers run on AWS Spot Instances for cheap, scalable clusters.<p>2. Pseudo-masterless architecture where clients read/write directly to the workers for true horizontal scaling + low latency (implemented in Golang).<p>In order to pull this off, we have also developed:<p><pre><code> - automatic worker replacement with backfill through Kafka
- spot instance price and stability prediction for optimal instance selection and pre-emptive replacement
- connection pooling w/ pgbouncer
- zero downtime cluster scaling (adding/removing workers, rebalancing table-shards - things Citus only includes in their Enterprise fork).
</code></pre>
We would love to develop an Open Source service that others can use and contribute to if there is interest. Would love to answer any questions!
My Guess is that 99% of the companies do not need such a solution. But 1% does. And if you target them, you have a business. Unfortunately that 1% does not frequent here (I guess).
(for regular workloads) cost of maintaining such a DB >> cost of RDS<p>It's a very interesting problem from a technical standpoint nonetheless.