Ask HN: Wouldn't it be cool to have a Supabase for SQLite?

All the features you requested Turso can do, no?<p>&gt; - SQLite runs next to server as production database<p>Embedded Replicas[1]<p>&gt; - In dev, some sort of worker auto-copies the prod DB to the local repo. Production bugs can be reproduced easily and code fixed quickly<p>Maybe not exactly this, but you can replicate a DB from another with the CLI tool. Then just use that as a dev db.<p>&gt; - Have an interface that lets you access, view, and modify data in the production DB, kind of like Firebase. Might need a server of its own... or couldn&#x27;t that just be the app server itself?<p>I don&#x27;t use their web interface, but I think it does allow this? I don&#x27;t know for sure though.<p>&gt; - SQLite auto-backs up to a bucket (like Litestream)<p>Turso does have point-in-time recovery[2]<p>[1] <a href="https:&#x2F;&#x2F;docs.turso.tech&#x2F;features&#x2F;embedded-replicas&#x2F;introduction" rel="nofollow">https:&#x2F;&#x2F;docs.turso.tech&#x2F;features&#x2F;embedded-replicas&#x2F;introduct...</a> [2] <a href="https:&#x2F;&#x2F;docs.turso.tech&#x2F;features&#x2F;point-in-time-recovery" rel="nofollow">https:&#x2F;&#x2F;docs.turso.tech&#x2F;features&#x2F;point-in-time-recovery</a>

It&#x27;s an obvious question, but have you looked into Pocketbase?<p><a href="https:&#x2F;&#x2F;github.com&#x2F;pocketbase&#x2F;pocketbase">https:&#x2F;&#x2F;github.com&#x2F;pocketbase&#x2F;pocketbase</a>

I&#x27;ve been keeping a fork of MVSqlite (sqlite on foundationdb) maintained.<p>What is Mvsqlite? According to the author it&#x27;s a distributed, MVCC SQLite that runs on top of FoundationDB.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;V-Sekai&#x2F;mvsqlite">https:&#x2F;&#x2F;github.com&#x2F;V-Sekai&#x2F;mvsqlite</a><p>Made an Elixir client, a Godot Engine client and CLI.<p>Supabase on Mvsqlite would be great! I am still puzzling over Mvsqlite&#x27;s write performance, but its read performance makes sense to me.<p># ycsb (workloadf, 10000, --wire-zstd)<p>Run finished, takes 1m18.086881433s<p>READ - Takes(s): 78.1, Count: 99957, OPS: 1280.3, Avg(us): 27061, Min(us): 1409, Max(us): 98751, 99th(us): 59391, 99.9th(us): 81791, 99.99th(us): 94591<p>READ_MODIFY_WRITE - Takes(s): 78.0, Count: 50061, OPS: 641.6, Avg(us): 69636, Min(us): 13320, Max(us): 205823, 99th(us): 121215, 99.9th(us): 189823, 99.99th(us): 203007<p>UPDATE - Takes(s): 78.0, Count: 50095, OPS: 642.0, Avg(us): 42630, Min(us): 7792, Max(us): 200319, 99th(us): 81535, 99.9th(us): 175487, 99.99th(us): 195199<p>From the github actions tests the original author wrote.

Working on it! =&gt; <a href="https:&#x2F;&#x2F;www.airsequel.com" rel="nofollow">https:&#x2F;&#x2F;www.airsequel.com</a>

Hey thenorthbay,<p>I gotta say, your idea of a Supabase-like system for SQLite got me thinking. It&#x27;s an interesting concept. SQLite running next to your server could certainly speed things up, but don&#x27;t forget about potential issues with concurrent writes. SQLite might not be as graceful as other databases in handling those.<p>The dev-prod synchronization you mentioned is a neat idea. Just remember to keep things secure and don&#x27;t expose sensitive data. Also, think about the size of your production database. If it&#x27;s too big, this might not be the best way to go.<p>Having an interface to access, view, and modify data straight from the production DB would be super useful. Depending on how complex this gets, you could either make it part of the application server or set up a separate server.<p>Auto-backup to a bucket is definitely a thumbs up from me. Tools like Litestream can be a lifesaver here.<p>Have you heard of Datasette? It&#x27;s an open-source tool that lets you explore and publish data from SQLite databases. It could be a good starting point for what you&#x27;re trying to build.<p>Your idea sounds like it could really shake things up. Keep us posted on how it goes. Good luck!

So like a normal SQL-server (PostgreSQL, MariaDB..), but with SQLite as backend. Which makes moving data simpler, as you only need to copy a file and can thus utilize the whole shell scripting-power. Well, seems like a nice idea for the dev, but how well can SQLite perform in a real production-environments with parallel writing? The only solutions I&#x27;ve seen so far are either relative poor, or &quot;cheat&quot; by using a separate file for each customer, which is not working with all use cases.<p>Anyway, there is a new generation of database-systems&#x2F;frontends, like Grist, NocoDB, Airtable, etc. But they are more focused on a user-friendly all round experience, not so much a good experience. But some of them do support working with SQLite.

Leaderless, eventually consistent DB, with DB running on same servers as app.<p>Could be good, you need to rethink all the distributed DB stuff.<p>Does a node deal with certain subset (e.g. certain customers) so you use customerid as a partition key.<p>CRDTs?<p>Once a node fills up to say x% capacity do you split that out (using consistent hashing to minimize effect of such spits).<p>You probably end up reinventing one of the distributed DBs once you make it production ready.<p>I think it is a lot of work unless you have simplifying requirements e.g. just give each customer a seperate VM and then use a mounted cloud storage to scale up space to some max like 10Tb then maybe cap it there. Write your db log to s3 glacier for a backup.<p>Second replica for failover.

Not directly an answer, but similar what if thinking got me wondering: wouldn&#x27;t it be cool to be able to use postgres much like how you use sqlite with python? I implemented this idea as a pip-installable python package, <a href="https:&#x2F;&#x2F;github.com&#x2F;orm011&#x2F;pgserver">https:&#x2F;&#x2F;github.com&#x2F;orm011&#x2F;pgserver</a>, and your feedback would be great :) I use it for my projects.

If I understand correctly, the SQLite library would be embedded in a sidecar processes that would run on the same machine as the app main process, instead of having the SQLite library embedded directly in the app main process? But then, what&#x27;s the point of using SQLite? What makes SQLite reads and writes so fast is that it is running embedded in the app process. If you don&#x27;t need that, and you want a client-server architecture, then you can simply run PostgreSQL or MySQL on the same machine, also as a sidecar process (or as a subprocess of the main process), and communicate using UNIX domain socket.

Certainly interested in having more solutions helping use SQLite. At dstack.ai, we use SQLite and love it.<p>- For our hosted version, we use Litestream; we lack a UI for accessing data.

SQLite reads and writes are already very fast, in particular because it&#x27;s running in process.<p>The minute you put it behind a socket, you lose that benefit. Also you would have to implement an efficient wire protocol.<p>I just don&#x27;t see the added value here.

Cloudflare D1 maybe? <a href="https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;d1&#x2F;" rel="nofollow">https:&#x2F;&#x2F;developers.cloudflare.com&#x2F;d1&#x2F;</a>

for dev there is already pg-lite server (postgres as nodejs&#x2F;wasm) you can spin up, it&#x27;s self-contained single-folder db to disk, 100MB RAM thingy. Highly experimental. Not sure if supabase codebase is setup for debugging the way you suggest.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;kamilogorek&#x2F;pglite-server">https:&#x2F;&#x2F;github.com&#x2F;kamilogorek&#x2F;pglite-server</a>

Pocketbase sounds like what you&#x27;re describing.

Exactly what you describe but not free [1]<p>1. <a href="https:&#x2F;&#x2F;subzero.cloud&#x2F;" rel="nofollow">https:&#x2F;&#x2F;subzero.cloud&#x2F;</a>

You can do all of this with Postgres.

All those features sound simple to do by hand, so if it doesn’t exist already, you should make it.

That&#x27;s precisely what we are building at <a href="https:&#x2F;&#x2F;sqlitecloud.io" rel="nofollow">https:&#x2F;&#x2F;sqlitecloud.io</a>

So… pocketbase.