Self-plug. This is based off the Haystack paper: <a href="https://www.usenix.org/legacy/event/osdi10/tech/full_papers/Beaver.pdf" rel="nofollow">https://www.usenix.org/legacy/event/osdi10/tech/full_papers/...</a> from Facebook. Relevant blog-post is here: <a href="https://code.facebook.com/posts/685565858139515/needle-in-a-haystack-efficient-storage-of-billions-of-photos/" rel="nofollow">https://code.facebook.com/posts/685565858139515/needle-in-a-...</a><p>I gave a talk about Haystack at PWL:
<a href="http://www.meetup.com/papers-we-love-too/events/220795812/" rel="nofollow">http://www.meetup.com/papers-we-love-too/events/220795812/</a> with the relevant video here:
<a href="https://www.youtube.com/watch?v=EuNumdi1Do0" rel="nofollow">https://www.youtube.com/watch?v=EuNumdi1Do0</a>
Looks very interesting, thanks. Kind of hilarious he didn't make a native Go API though..<p>This is the only one AFAICT:
<a href="https://github.com/ginuerzh/weedo" rel="nofollow">https://github.com/ginuerzh/weedo</a>
Looks interesting. I'm curious about why there's a 32bit cookie added to the volume,filename -- it seems like a rather weak protection -- and as such, it becomes an unnecessary complication? As have been shown with facebook, relying on (permanent) secret urls to grant/deny access is a bad idea.<p>So, why not just use volume,id, and then deploy a proxy that handless access based on tokens in front -- if access control is wanted? (Not all uses of files will need/want access control).<p>I suppose one reason for a "cookie" would be cache invalidation in case of volume,id reuse.
I'm thinking that IPFS (<a href="http://ipfs.io/" rel="nofollow">http://ipfs.io/</a>) has more of an opportunity to fill the distributed-filesystem role de jeur, alas .. Seaweed-FS seems flakey and not quite ready for primetime. Poor conclusion?