TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

SlateDB – An embedded database built on object storage

162 pointsby notamy8 months ago

14 comments

nmca8 months ago
&gt; Object storage is an amazing technology. It provides highly-durable, highly-scalable, highly-available storage at a great cost.<p>I don’t know if this was intended to be intentional funny, but there is a little ambiguity in the expression “great cost”, typically great cost means very expensive.<p>Very cool and useful shim otherwise :)
评论 #41721881 未加载
评论 #41721160 未加载
评论 #41723669 未加载
drodgers8 months ago
It looks like writes are buffered in an in-memory write ahead log before being written to object storage, which means that if the writer box dies, then you lose acknowledged writes.<p>I&#x27;ve built something similar for low-cost storage of infrequently accessed data, but it uses our DBMS (MySQL) for the WAL (+ cache of hot reads), so you get proper durability guarantees.<p>The other cool trick to use is to use Bε-trees (a relatively recent innovation from Microsoft Research) for the object storage compaction to minimise the number of write operations needed when flushing the WAL.
评论 #41720291 未加载
评论 #41722735 未加载
rehevkor58 months ago
I don&#x27;t see how it&#x27;s embedded if it relies on nonlocal services... on the contrary it says specifically, &quot;no local state&quot;. It appears to be more analogous to a &quot;lakehouse architecture&quot; implementation (similar to, for example, Apache Iceberg), where your app includes a library that knows how to interact with the data in cloud object storage.
评论 #41724490 未加载
anon2918 months ago
This seems to be a key value store built atop object storage. Which is to say, it seems completely redundant. Not sure if there&#x27;s some feature I&#x27;m missing, but all of the six features mentioned on the front page are things you&#x27;d have if you used the key value store directly (actually, you get more because then you get multiple writers).<p>I was excited at first and thought this was SQL atop S3 et al. I&#x27;ve jerryrigged a solution to this using SQLite with a customized VFS backend, and would suggest that as an alternative to this particular project. You get the benefit of ACID transactions across multiple tables and a distributed backend.
评论 #41716354 未加载
评论 #41715581 未加载
评论 #41715348 未加载
评论 #41716179 未加载
评论 #41715303 未加载
jitl8 months ago
From the docs <a href="https:&#x2F;&#x2F;slatedb.io&#x2F;docs&#x2F;introduction&#x2F;" rel="nofollow">https:&#x2F;&#x2F;slatedb.io&#x2F;docs&#x2F;introduction&#x2F;</a><p>&gt; NOTE<p>&gt; Snapshot isolation and transactions are planned but not yet implemented.
评论 #41720315 未加载
remon8 months ago
I&#x27;ve read the introduction and descriptions two times now and I still don&#x27;t understand what this adds to the proceedings. It appears to be an extremely thin abstraction over object storage solutions rather than an actual DB which the name and their texts imply.
yawnxyz8 months ago
is this an easier to do the &quot;store parquet on s3 &gt; stream to duckdb&quot; pattern that&#x27;s popping up more and more?
评论 #41716160 未加载
评论 #41718345 未加载
评论 #41716170 未加载
shenli35148 months ago
Went thru the document: <a href="https:&#x2F;&#x2F;slatedb.io&#x2F;docs&#x2F;introduction&#x2F;#use-cases" rel="nofollow">https:&#x2F;&#x2F;slatedb.io&#x2F;docs&#x2F;introduction&#x2F;#use-cases</a> I can not understand why are they targeting the following use cases with this architecture. * Stream processing * Serverless functions * Durable execution * Workflow orchestration * Durable caches * Data lakes
hantusk8 months ago
Since writes to object storage are going to be slow anyway, why not double down on read optimized B-trees rather than write optimized LSM&#x27;s?
评论 #41717844 未加载
评论 #41722800 未加载
epolanski8 months ago
Not a db guy, just asking, what does it mean &quot;embedded&quot; database?<p>I&#x27;m confused here, because Google says it&#x27;s a db bundled with the application, but that&#x27;s not really what I get from the landing page.<p>What problem does it solve?
评论 #41715908 未加载
loxias8 months ago
Can I please, please, please, have C++ or at least C bindings? :) Or the desired way to call Rust from another runtime? I don&#x27;t know any Rust.
评论 #41716304 未加载
demarq8 months ago
Embed cloud<p>Sounds like they just cancel each other out. Not sure what advantage embedding will yield here
goodpoint8 months ago
Despite the name this is not a database.
评论 #41719256 未加载
tgdn8 months ago
&quot;It doesn&#x27;t currently ship with any language bindings&quot;<p>Rust is needed to use SlateDB at the moment