TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

K4 – High performance open-source transactional, durable embedded storage engine

35 点作者 alexpadula6 个月前

7 条评论

alexpadula6 个月前
Hello everyone! I hope you are all well.<p>I&#x27;ve been working on a few storage engine designs recently implementing an LSM tree and would like to share K4.<p>K4 is a storage engine written completely in GO with no dependencies. The write speed surpasses RocksDB (7.8.3) with similar configuration.<p>I will write way more comprehensive benchmarks down the line.<p>The current features of K4<p>------------------------------<p>- High speed writes and reads<p>- Durability<p>- Variable length binary keys and values. Keys and their values can be any length<p>- Write-Ahead Logging (WAL). System writes PUT and DELETE operations to a log file before applying them to the LSM tree.<p>- Atomic transactions. Multiple PUT and DELETE operations can be grouped together and applied atomically to the LSM tree.<p>- Paired compaction. SSTables are paired up during compaction and merged into a single SSTable(s). This reduces the number of SSTables and minimizes disk I&#x2F;O for read operations.<p>- Memtable implemented as a skip list.<p>- In-memory and disk-based storage<p>- Configurable memtable flush threshold<p>- Configurable compaction interval (in seconds)<p>- Configurable logging<p>- Configurable skip list<p>- Bloom filter for faster lookups. SSTable initial pages contain a bloom filter. The system uses the bloom filter to determine if a key is in the SSTable before scanning the SSTable.<p>- Recovery from WAL<p>- Granular page locking<p>- Thread-safe<p>- TTL (time to live) support<p>- Optional compression support (Simple lightweight and optimized Lempel-Ziv 1977 inspired compression algorithm<p>- Range and equi functionality (Get, NGet, Range, NRange, GreaterThan, GreaterThanEq, LessThan, LessThanEq)<p>- No dependencies<p>I am in the process of writing a C binding for K4 which will enable me to write FFI&#x27;s for multiple other languages like Python, Node.JS, Ruby, etc.<p>I hope you get a chance to check it out and do let me know your thoughts!<p>Thank you kindly.
评论 #41992455 未加载
评论 #41993394 未加载
评论 #41992894 未加载
评论 #41993185 未加载
jitl6 个月前
It’s cool to see new storage engines, thanks for sharing.<p>For transaction, how are conflicts on concurrent transaction handled? What’s the ordering? Are you considering any conditional check operation like compare-and-swap? For example I think Deno KV’s API where each value has a versionstamp and for transactions there’s an operation CHECK(key, versionstamp) that cancels the transaction if key doesn’t have the given versionstamp at the time of transaction.
评论 #41996181 未加载
fithisux6 个月前
Keep it coming. I love the work people put on storage engines.<p>Do you believe a graph storage can be layered on top of K4?
评论 #41997288 未加载
评论 #41997337 未加载
alexpadula6 个月前
Just putting this out there. I will be writing a paper on my experiences writing storage engines and I will be releasing a full website for K4 at k4db.com<p>Cheers all
knowitnone6 个月前
Nice work! Any reason why Go was used and not C or Rust? I was under the impression Go would be slower and you are aiming for high performance.
评论 #41996999 未加载
Gys6 个月前
This should be a ‘Show HN’?
评论 #41994568 未加载
0x33316 个月前
Insane.
评论 #42043215 未加载