We're having lots of trouble getting to many AWS/Amazon resources in US-West2 and it seems like many other people are reporting problems with US-East. Currently we cannot even reach http://status.aws.amazon.com. What are other people seeing and does anyone have any pointers to what the problem could be? There seems to also be a bit of discussion on Twitter: https://twitter.com/hashtag/aws?src=hash&vertical=default&f=tweets#
I often find out about these sorts of global internet issues early because of a little project of mine:<p><a href="https://statusgator.io" rel="nofollow">https://statusgator.io</a><p>It just monitors the status pages of lots of different services. When I get 30 notices from various unrelated services at once all with comments like "Network connectivity issues", I know some kind of routing issue is plaguing the 'net.
Looks to be because of: <a href="https://twitter.com/Axcelx/status/616058414746202113" rel="nofollow">https://twitter.com/Axcelx/status/616058414746202113</a>
I run a spam filtering company in AWS-Oregon and we are under a DDoS that is directed to some of our financial services clients...We know that our client is the target because the Russian team claims responsibility opened up a trouble ticket with us (not hosted in AWS) about 2 hours ago to let us know that they would DDoS us and the network.<p>I am sure it's just a coincidence :-)
There's another post on HN:<p>"EC2 us-east is down" - <a href="https://news.ycombinator.com/item?id=9809304" rel="nofollow">https://news.ycombinator.com/item?id=9809304</a>
"Between 5:25 PM and 6:07 PM PDT we experienced an Internet connectivity issue with a provider outside of our network. The issue has been resolved and the service is operating normally. "<p><a href="http://status.aws.amazon.com/" rel="nofollow">http://status.aws.amazon.com/</a><p>I really hate how AWS use PDT for their status page. Seriously, is there any logical reason to use anything other than UTC? I find it means when there is a report, I first need to convert the PTD time shown to UTC, then to local time just so I can know if the issue is possibly the one I experienced. Unless you're also in PDT, couldn't they just save everyone else one step by showing all service issues in UTC?
Note at the top of <a href="http://status.aws.amazon.com/" rel="nofollow">http://status.aws.amazon.com/</a>:<p>Internet connectivity issues<p>We are currently monitoring an external Internet provider issue that is causing interrupted service connectivity to AWS services for some customers. AWS services are not affected and continue to operate normally.
I can't get the status page to load: <a href="http://status.aws.amazon.com/" rel="nofollow">http://status.aws.amazon.com/</a><p>Edit: Scratch that. Some stuff I'm hosting in AWS East is still working. Maybe someone screwed up router configs on the west coast.<p>Edit: Rollback the scratch. It's oscillating between up and down right now.
Status page updated:<p>"We are currently monitoring an external Internet provider issue that is causing interrupted service connectivity to AWS services for some customers. AWS services are not affected and continue to operate normally."<p><a href="http://status.aws.amazon.com/" rel="nofollow">http://status.aws.amazon.com/</a>
I woke up to see two of my instances, running in AWS Singapore, at 100% and unresponsive. Rebooting also did not help. They had to be killed. Thankfully AWS had documented this on their support page:<p>"Additionally, some customers have reported continued connectivity issues for some of their instances. We have seen with these reported issues that this has been caused by a leap second bug within the instance operating system, which results in 100% CPU utilization. We recommend rebooting the instance via the EC2 Management Console or API, or resetting the operating system time to resolve the issue. For further information see:<p><a href="https://access.redhat.com/articles/15145"" rel="nofollow">https://access.redhat.com/articles/15145"</a>
A fiber cut in Oregon could be responsible (<a href="https://puck.nether.net/pipermail/outages/2015-June/007906.html" rel="nofollow">https://puck.nether.net/pipermail/outages/2015-June/007906.h...</a>). I've been seeing connectivity issues with us-west-2 for most of the day.<p>There was also a fiber cut in San Francisco area this morning (<a href="http://www.usatoday.com/story/tech/2015/06/30/california-internet-outage/29521335/" rel="nofollow">http://www.usatoday.com/story/tech/2015/06/30/california-int...</a>).
I can reach <a href="http://status.aws.amazon.com/" rel="nofollow">http://status.aws.amazon.com/</a> from my home Comcast connection in SF. The status page is showing green for all services.
Looks to be something strange with the network. Using a SOCKS proxy to a host running on he.net, I can get to US-East machines, but not from my Cogent-based connection in the office.
For what its worth, I usually start at [1].<p>[1] <a href="http://internetpulse.keynote.com/" rel="nofollow">http://internetpulse.keynote.com/</a>
I've been having issues all week reaching a lot of different AWS websites including atom.io.<p>Switching my DNS to 8.8.8.8 fixed it for me. Something is up though.