TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

How long will a 64 bit Transaction-ID last in PostgreSQL?

101 点作者 vinayan3超过 6 年前

8 条评论

anarazel超过 6 年前
There&#x27;s no chance we go for 64bit transaction ids on the tuples themselves - the space increase would be far too big. The overhead of tuple headers is already a problem, and xmin&#x2F;xmax are a significant portion of that.<p>There were patches however that kept an &#x27;epoch&#x27; (the upper 32bit of a 64bit transaction id) on a page level. Plus some rewrite logic when transactions that are too far away from each other to be represented as an index from a base epoch are about to be present on one page. That&#x27;d allow to effectively have 64bit xids.<p>The in-development zheap storage engine basically does something roughly akin to that, removing the need to perform freezing when a table becomes older than ~2^31 - safety-window transactions.<p>The transaction id that the system internally has effectively already keeps track of of xids in a 64bit manner, albeit in a somewhat over-complicated manner by keeping track of an epoch separately (there&#x27;s a patch likely to land in the next version to just go for 64bit there). That&#x27;s why you can see e.g. txid_current() return 64bit transaction ids.
评论 #19084229 未加载
评论 #19084108 未加载
amarshall超过 6 年前
This seems to be (with coincidental timing) the cause of Mandrill’s current outage [1]:<p>&gt; Mandrill uses a sharded Postgres setup as one of our main datastores. On Sunday, February 3, at 10:30pm EST, 1 of our 5 physical Postgres instances saw a significant spike in writes. The spike in writes triggered a Transaction ID Wraparound issue. When this occurs, database activity is completely halted. The database sets itself in read-only mode until offline maintenance (known as vacuuming) can occur.<p>&gt; The database is large—running the vacuum process takes a significant amount of time and resources, and there’s no clear way to track progress.<p>[1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=19084525" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=19084525</a>
评论 #19086920 未加载
throwawaymath超过 6 年前
The author talks a bit about the architecture of PostgreSQL transactions, touching on lazy transaction ID consumption and vacuuming. Notably, writes require IDs but reads do not. So this is focused on write-optimized workloads.<p>If you want to get the basic tl;dr which answers the headline: these IDs will last so long it’s almost not worth quantifying. This is an obvious calculation even if you assume ostentatatious performance requirements three orders of magnitude greater than the author’s:<p><pre><code> 2^64 &#x2F; (86,000 * 1,000,000,000) = 213,503.9 </code></pre> The author uses 1,000,000 writes&#x2F;second; I prefer 1,000,000,000 since it’s more ridiculous. There are 86,000 seconds in a day. It will take you the better part of a millenium to exhaust those IDs, assuming you consume an average of one billion every single second.<p>The author didn’t talk about collisions, but those are worth mentioning because you could even confidently assign these randomly instead of incrementally. Since a collision will occur (in expectation) after 2^63 transactions, you shouldn’t even have to worry about a single one occuring (on average) for almost 300 years.<p>Of course, using 64-bit IDs comes with nontrivial space increase - every single tuple will increase by a factor of 2.<p>EDIT: Original collision estimate is wrong, see corrections. I took (2^n)&#x2F;2 = 2^(n-1) as the birthday bound instead of 2^(n&#x2F;2).
评论 #19083346 未加载
评论 #19083281 未加载
评论 #19083382 未加载
评论 #19083471 未加载
aaronbwebber超过 6 年前
I highly recommend using flexible-freeze if you run Postgres in production - does not take very much effort to set up and almost certainly will help you avoid issues with txnid wraparound:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;pgexperts&#x2F;flexible-freeze" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;pgexperts&#x2F;flexible-freeze</a><p>It just runs `VACUUM FREEZE` when you schedule it (usually daily), starting with the tables closest to hitting a wraparound.
kostaw超过 6 年前
Just imagine running `VACUUM` on that table that wrote 1M rows&#x2F;seconds for 300 years and now you need to vaccuum quick because the transaction ids will wrap around next year...
评论 #19084113 未加载
Thorrez超过 6 年前
What if we interpret Moore&#x27;s law to say that transaction speed will double every 2 years?
评论 #19083939 未加载
评论 #19084655 未加载
评论 #19085573 未加载
hyperman1超过 6 年前
What I don&#x27;t understand is how this only affects postgress. How do db2&#x2F;mssql&#x2F;oracle handle mvcc? Is it superior or is it a case of trade offs? Supposing the answer is publicly available.
xurukefi超过 6 年前
I guess people had similar arguments when designing IPv4.