TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Data Mesh: The Answer to the Data Warehouse Hypocrisy

25 点作者 electrum大约 4 年前

2 条评论

hashhar大约 4 年前
A golden nugget right at the beginning of the article.<p>&gt; <i>There is one rule, pretty much the only rule, that determines success in scaling data systems: avoid coordination whenever possible. This one rule neatly summarizes my entire career until this point --- the quarter century I’ve spent teaching, innovating, and building scalable data systems. At the end of the day it all comes down to one thing --- avoiding coordination.</i><p>In one of my previous jobs there was an inventory management service which was broken into two layers - one service that kept track of the inventory at a particular warehouse (including the location and more specific details) and another one which tracked the overall inventory across the entire system (without the specific details).<p>Whenever an inventory had to be allocated there was a need to co-ordinate between the global and local inventory systems. Easy enough to do with optimistic locking and performing compensation actions. But it turned into a performance bottleneck really soon.<p>Deciding to replace that system with an immutable ledger which gets lazily materialized improved throughput by a lot. You obviously had to handle the cases where the stale materialization meant that you&#x27;d just overcommited inventory but it was easy enough to do and allowed us to scale up the peak order processing rate by 6x.
abadid大约 4 年前
I&#x27;m the author of this piece. I&#x27;m happy to respond to comments in this thread.
评论 #26581688 未加载