How Facebook stores its billions of photos

That was much more exhaustive than i was anticipating. I'm curious how this stacks up to sites like flickr.

I wonder if they have some mechanism for rebuilding the raid array periodically to handle the inevitable undetected read errors. With raid6 you can actually do this, by comparing both parity blocks with the actual data.<p>Hard disks typically have undetected read error rates of 1 bit in 1E15, so assuming they transfer 1PB per day that's about 9 errors per day. Which isn't much I guess, but I wonder if they do anything about it.

Here's a Flowgram presentation from last year, "Needle in a Haystack", which covers these updates. <a href="http://www.flowgram.com/f/p.html#2qi3k8eicrfgkv" rel="nofollow">http://www.flowgram.com/f/p.html#2qi3k8eicrfgkv</a>

Why bother putting the haystack in a filesystem? Just use a raw partition.<p>Haystack effectively IS a filesystem - a log structured filesystem.

raw io? facebook better go ipo soon, they're running out of abstraction layers to code through