TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

How SQLite scales read concurrency

240 pointsby pkilgoreover 2 years ago

9 comments

simonwover 2 years ago
Something I found non-obvious about WAL mode in SQLite is that it&#x27;s actually a property of the database file itself.<p>When you run &quot;PRAGMA journal_mode=wal;&quot; against a database file the mode is permanently changed for that file - and the .db-wal and .db-shm files for that database will appear in the same directory as it.<p>Any future connections to that database will use it in WAL mode - until you switch the mode on it back, at which point it will go back to journal mode.<p>It makes sense when you think about it - of course a database can only be in one or the other modes, not both, so the setting must be at the database file level. But it took me a while to understand.<p>I wrote some notes on this here: <a href="https:&#x2F;&#x2F;til.simonwillison.net&#x2F;sqlite&#x2F;enabling-wal-mode" rel="nofollow">https:&#x2F;&#x2F;til.simonwillison.net&#x2F;sqlite&#x2F;enabling-wal-mode</a>
评论 #32585414 未加载
评论 #32581486 未加载
评论 #32585836 未加载
Syttenover 2 years ago
Also worth noting that Sqlite released a WAL2 journal mode recently that eliminates the &quot;stop of the world&quot; we had with WAL when checkpointing. Basically it maintains two wal files and switched between them when one needs to be checkpointed. It is quite neat!
评论 #32581639 未加载
评论 #32585213 未加载
etaioinshrdluover 2 years ago
It&#x27;s become popular to talk about how scalable SQLite is lately, but let&#x27;s not forget the elephant in the room, it only allows a single writer at a time.<p>It&#x27;s obviously easier to manage and maintain due to it being an embedded database, but that seems to be a very separate issue from the data structure involved, which definitely has disadvantages compared to a typical SQL system.
评论 #32586306 未加载
评论 #32586100 未加载
评论 #32583740 未加载
评论 #32589009 未加载
simonwover 2 years ago
&quot;The database header has a read &amp; write version at bytes 18 &amp; 19, respectively, that are used to determine the journal mode. They&#x27;re set to 0x01 for rollback journal and 0x02 for write-ahead log. They&#x27;re typically both set to the same value.&quot;<p>Are there any useful situations where they wouldn&#x27;t be set to the same value?
posharmaover 2 years ago
What&#x27;s with so many SQLite posts on HN these days? Never saw so much love for sqlite before.
评论 #32584862 未加载
评论 #32584151 未加载
评论 #32588874 未加载
jef_leppardover 2 years ago
Someone recently observed that SQLite would be used a lot more in production if it didn&#x27;t have the word &quot;lite&quot; in its name. I&#x27;ve personally been amazed by what it can do. Really great piece of tech with an unfortunate name.
评论 #32581855 未加载
评论 #32581117 未加载
评论 #32587456 未加载
评论 #32581642 未加载
评论 #32588935 未加载
评论 #32581663 未加载
评论 #32587223 未加载
评论 #32581631 未加载
NeutralForestover 2 years ago
Fly.io constantly has good articles about databases, really cool to see it.
christophilusover 2 years ago
SQLite is such an excellent little tool.<p>I recently ran an experiment with Node and was able to take writes from around 750 &#x2F; second to around 24K &#x2F; second just by coordinating them using Node IPC. That is, I had the main thread own the sole write connection and all other threads sent their writes operations to it, thread-local connections were read-only.<p>It&#x27;s pretty cool how far you can push SQLite and it just keeps humming right along.
评论 #32581943 未加载
ithrowover 2 years ago
SQLite needs some &#x27;put your money where your mouth is&#x27; benchmarks.<p>EDIT:Lots of devs have (unfounded) doubts about performance regarding SQLite for web apps, a single machine benchmark against postgres (with postgres and the app in the same server) would clear many doubts and create awareness that SQLite is going to be more than enough for many apps.. The app doesn&#x27;t have to be a web app (we are not testing web servers) but maybe some code with some semi-complex domain models.
评论 #32582869 未加载
评论 #32583089 未加载
评论 #32585658 未加载
评论 #32582896 未加载