><i>Randomise your bucket names! There is no need to use company-backup.s3.amazonaws.com</i><p>This is really poor advice. It offers no real benefit, especially since any asset you access will betray your bucket name because it's part of the DNS resolution. Bucket names are emphatically public as much as a DNS name is public.
Could be more general: finding subdomains by watching CT logs.<p>So what is the problem here?<p>How to "hide" private subdomains?<p>How to "securely" configure S3 buckets?<p>IMO, the problem is in the use of the CA system, where control over "names" (e.g. subdomains) is shared with third parties (certificate issuers) instead of being solely with the user who wants to reserve names.<p>It is possible to have a non-CA PKI system where the user controls both the issuance of the public key <i>and</i> the associated name she will use. In such a system, no third party has control over names. People learn the user's name and the user's key from the same source: the user.<p>Thus there is no issue of trust re: using third parties, and thus no need for monitoring what names the third parties are issuing, e.g. via "certificate transparency" logs. CT logs do not need to exist.<p>This is not a new idea and it has been proven to work. I can prepare a post with examples if anyone is interested.
> Randomise your bucket names! There is no need to use company-backup.s3.amazonaws.com.<p>I don't think this is a globally true statement. Random bucket names are hard, not everyone is using s3 with a code configuration and therefore remembering bucket name is actually important.
Passive DNS might be another good way to get S3 bucket names.<p>There doesn't seem to be a Wikipedia article on Passive DNS, but this article explains it quite well: <a href="https://help.passivetotal.org/passive_dns.html" rel="nofollow">https://help.passivetotal.org/passive_dns.html</a><p>Basically some resolvers submit all (some?) of their DNS query responses to a central database so that it can be searched later. It seems you can also install a passive "sensor" in your network that (presumably) passively MITMs DNS queries and then sends off the responses.<p>I don't know how hard it is to get access to the data, but:<p>> programs like RiskIQ's DNSIQ allow organizations to install a sensor on their network that reports back to RiskIQ and in exchange, the organization gains access to all the passive DNS traffic inside the central repository.<p>EDIT: VirusTotal has some passive DNS data publicly available: e.g. look in "observed subdomains" <a href="https://www.virustotal.com/en/domain/s3-us-west-2.amazonaws.com/information/" rel="nofollow">https://www.virustotal.com/en/domain/s3-us-west-2.amazonaws....</a><p>EDIT2: And a bunch of them appear to be unprotected...
I did some analysis a few months ago and collected the names of approximately 100,000 buckets in the wild. Rough numbers, about 5% are open to the public for anonymous read, and about 5% of those are open for anonymous write.<p>I'm convinced that Chris Vickery, the guy behind a good many of the open bucket finds this year, has access to enterprise firewall/proxy logs. Not because the buckets would have been hard to find, but because you could spend a lifetime looking through thousands upon thousands of open buckets before you find anything interesting.
Love stuff like that! I've quickly wrapped a prettifier for S3 xml listings in a userscript, so you can use it with Tampermonkey Beta. Tested on Chrome under OS X.<p><a href="https://gist.github.com/kaivi/8114cbc2080da78d67c94238af64210d" rel="nofollow">https://gist.github.com/kaivi/8114cbc2080da78d67c94238af6421...</a><p>Edit: Okay, the userscript won't run on larger XML files, gotta figure it out later.
This is concerning b/c there have been a number of high profile data breaches that have occurred due to over reliance on S3 bucket obscurity. Where the buckets have been left with minimal or misconfigured permissions and GBs of data there for the downloading.
I was curious so I've tried if I could find anything compromising with it and it's mostly just public buckets of some images used for websites so nothing strange. Maybe the README is a bit too dramatic.