TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Amazon’s Glacier secret: BDXL (2014)

213 点作者 another超过 8 年前

20 条评论

fnord123超过 8 年前
They&#x27;re almost certainly doing something like Microsoft&#x27;s Pelican: <a href="https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;2016&#x2F;09&#x2F;pelican-hotstorage2016.pdf" rel="nofollow">https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;...</a><p>The first comment on TFA says as much.<p>Edit: This is the actual Pelican paper: <a href="https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;2014&#x2F;10&#x2F;osdi2014-Pelican.pdf" rel="nofollow">https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;wp-content&#x2F;uploads&#x2F;...</a>
评论 #13406629 未加载
评论 #13406156 未加载
评论 #13409119 未加载
hemancuso超过 8 年前
I highly doubt this. Especially given the introduction of the near line tiers.<p>It&#x27;s probably just very widely striped, price-segmented data.<p>Also, see: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=4416065" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=4416065</a>
Spooky23超过 8 年前
I thought this was debunked at the time?<p>When I was running exchange systems, our biggest challenge was delivering IOPS. We had to use SAN, and wasted significant storage because we&#x27;d spend our IOPS budget at 40-60% storage capacity.<p>I figured at their scale they would have similar problems.
评论 #13405682 未加载
评论 #13409186 未加载
UseStrict超过 8 年前
Using BDXL seems like a pretty good solution. Most of this data is archival and existing data is very unlikely to change. You can use HDD&#x2F;SSD as a buffer as users upload data, and then optimize the packing to ensure you&#x27;re using all available space on a disk. Possibly encrypt each user&#x27;s data block on the disk. The system itself would only need to track metadata (file metadata, cartridge&#x2F;disk, key). Deleting a file would be deleting the key and marking the file as inactive. Once&#x2F;if a cartridge is marked as completely deleted, can just recycle it.
评论 #13410530 未加载
zitterbewegung超过 8 年前
Nice investigation and also Facebook has been using 50gb blue rays <a href="http:&#x2F;&#x2F;www.businessinsider.com&#x2F;facebook-uses-10000-blu-rays-for-backup-2014-1" rel="nofollow">http:&#x2F;&#x2F;www.businessinsider.com&#x2F;facebook-uses-10000-blu-rays-...</a> and is moving to 300gb <a href="http:&#x2F;&#x2F;www.businessinsider.com&#x2F;ces-2016-facebook-uses-panasonic-freezeray-2016-1" rel="nofollow">http:&#x2F;&#x2F;www.businessinsider.com&#x2F;ces-2016-facebook-uses-panaso...</a> .
评论 #13405235 未加载
评论 #13409093 未加载
评论 #13408183 未加载
shiftpgdn超过 8 年前
Why isn&#x27;t anyone thinking tapes? You can get LTO 7 tapes for $0.008 per Gigabyte that allow 100-300 writes before the tape should be destroyed. Quantum and HP make monstrous tape libraries that hold 5-10 petabytes per rack. You can also cartridge-ize your library for even more dense storage on a literal warehouse rack somewhere.<p>Tapes also match the slow retrieval speeds as you have to read the data out onto a drive linearly.
评论 #13412852 未加载
tyingq超过 8 年前
Previous HN discussion about this: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=7647571" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=7647571</a>
saosebastiao超过 8 年前
This is an extremely interesting deductive analysis. However, considering it is amazon, there always exists that persistent &quot;other&quot; possibility: they&#x27;re purposefully taking a loss.
评论 #13406150 未加载
WalterBright超过 8 年前
Given the scale of Glacier, I&#x27;m surprised that Amazon is able to keep their underlying storage technology a secret.
评论 #13406460 未加载
binaryanomaly超过 8 年前
Does someone know about Google nearline and coldline storage? Google claims coldline access within miliseconds.
评论 #13405565 未加载
mixmastamyk超过 8 年前
I&#x27;ve got a USB3 BDXL writer attached at my desk and it is quite handy and not too expensive. I back up my whole data (work) partition to it every so often and occasionally take one over to a relative&#x27;s house as my own home-grown &quot;glacier&quot; system.
评论 #13405854 未加载
评论 #13405688 未加载
评论 #13405986 未加载
Twirrim超过 8 年前
Ex-Glacier engineer... and no I&#x27;m not going to tell you what or how it&#x27;s done. NDAs and all that jazz. These speculation threads always make for fascinating reading for people on the team.
评论 #13407531 未加载
评论 #13408327 未加载
digi_owl超过 8 年前
Are there packet written?<p>That seems to have been the major stumbling block with higher capacity optical media, that one can&#x27;t do the drag and drop writes that one have with spinning rust and flash chips.
评论 #13405649 未加载
jayonsoftware超过 8 年前
What if they are using 2.5 inch 5TB drives like this <a href="http:&#x2F;&#x2F;www.theverge.com&#x2F;circuitbreaker&#x2F;2016&#x2F;11&#x2F;15&#x2F;13642078&#x2F;seagate-backup-plus-portable-5tb-hard-drive" rel="nofollow">http:&#x2F;&#x2F;www.theverge.com&#x2F;circuitbreaker&#x2F;2016&#x2F;11&#x2F;15&#x2F;13642078&#x2F;s...</a> I use. They are nice as we can plug them into a 15 port USB stick, they auto power down when not in used. Amazon could have developed a box like what backblaze.com has done.
kennethh超过 8 年前
Facebook disclosed how they are archiving photos long term, they manage a 1 to 1.4 ratio with Reed Solomon Redundancy and 8 disk of 14 can fail without loosing data. <a href="https:&#x2F;&#x2F;code.facebook.com&#x2F;posts&#x2F;1433093613662262&#x2F;-under-the-hood-facebook-s-cold-storage-system-&#x2F;" rel="nofollow">https:&#x2F;&#x2F;code.facebook.com&#x2F;posts&#x2F;1433093613662262&#x2F;-under-the-...</a>
KaiserPro超过 8 年前
From what I recall, Writable optical disks have a much shorter life span compared to tape (~15 years vs 75 years)<p>Plus, if I was designing an archival system, it wouldn&#x27;t be on blueray, unless there was a requirement for magnetic resistance.
评论 #13410327 未加载
评论 #13406409 未加载
gwicke超过 8 年前
I wonder if Amazon is also deduplicating. It seems likely that a share of users would store large media files or software packages without encryption.
Flammy超过 8 年前
I remember reading this back around when it came out. Have there been any new pieces of the puzzle identified (or announcements...) since then?
Nition超过 8 年前
Could it possibly just be compression on tradtional HDDs?<p>- Cheaper storage because data is heavily compressed<p>- Slow retrieval time due to slow decompression
评论 #13408038 未加载
rrggrr超过 8 年前
Custom engineered SSD. Powered off at rest.<p>See: <a href="http:&#x2F;&#x2F;www.storagesearch.com&#x2F;ssd-petabyte.html" rel="nofollow">http:&#x2F;&#x2F;www.storagesearch.com&#x2F;ssd-petabyte.html</a>
评论 #13410473 未加载