TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Updating a 50 terabyte PostgreSQL database (2018)

187 点作者 whitepoplar大约 4 年前

7 条评论

fastest963大约 4 年前
They're halting database traffic for 15-30 minutes. I'm confused how this won't mean they have downtime? The application can queue transactions but how is the application handling reads after those queued trsnactions and what if 2 queued transactions conflict?
评论 #26538283 未加载
评论 #26537755 未加载
评论 #26538245 未加载
评论 #26536291 未加载
评论 #26537725 未加载
评论 #26537862 未加载
pupdogg大约 4 年前
Based on their numbers, they should be archiving their historical data in parquet format partitioned by YYYYMMDD onto something like Clickhouse. This way, they can run a lean Postgres instance(s) at all times yet still get benefits of real-time reporting. Based on their use case, they can retain up-to 30 days of data in Postgres and offload the rest onto Clickhouse.
评论 #26539057 未加载
z3t4大约 4 年前
I would probably implement an "archive" system that stores static data that is mostly accessed by "sum" functions - In order to keep the main database relatively small. If the data is immutable/static replication and caching becomes much easier.
评论 #26538976 未加载
gdgdgdgbgfff大约 4 年前
Vertical scaling like this will have quite a few physical limits, eg max cores (is it 64?). I would think that they may be fast approaching these limits?. For a an scalable enterprise solution would it not be better, and also more cost-efficient, to horizontally scale. Shard the tables over many nodes.
评论 #26539609 未加载
janandonly大约 4 年前
Am I the only one who is shocked to discover that payments (even very small sums) are stored for years on end, maybe indefinite??<p>Simple deleting a row that is 366 days old is not an option to keep the PostgreSQL DB relatively small?
评论 #26538902 未加载
评论 #26538878 未加载
asguy大约 4 年前
(2018)
评论 #26537263 未加载
hardwaresofton大约 4 年前
&gt; As PostgreSQL options were not suitable for the next upgrade, in parallel we considered other possibilities. Our storage devices were able to make instant snapshots and also make them available on remote storage devices over the network, within a much smaller timeframe<p>ZFS?
评论 #26553033 未加载