AWS power failure in US-EAST-1 region killed some hardware and instances

Not piling on AWS. These things happen and I’m sure everyone involved is working to improve things. Yes, everyone should deploy in multiple availability zones.<p>My 2 cents. Outages happen. Network glitches happen. Bad configs and bad updates happen. However, power issues should not really happen. One of the primary cost saving areas of going to the cloud is not having to do on-prem power, such as UPS, generators, maintenance etc. Not having to do on-prem cooling is another thing. These should be solved from the customer’s perspective when going into a professional data center and are the things you don’t want to worry about anymore.

Any idea why a power <i>failure</i> would cause (or reveal) hardware damage?<p>Leading up to Y2K, I remember concerns about spinning hard disks not being able to start up again.<p>And if the power is <i>flaky</i> with spikes and brown-outs, I understand that&#x27;s a problem.<p>But is either of those relevant to AWS?

Have there been any incidents that affected more than one AZ?<p>AWS RDS in multi-AZ deployment gives you two availability zones. Aurora gives you three. What kind of scenario would be used to justify three AZ’s for the purposes of high availability?

Rare to see power issues at a modern data center cause downtime. All of those racks should have UPS and batteries to sustain during an outage until the automatic transfer switch can fail over to a redundant system or generator. Would be interested in reading more about what happened here.

Not an expert, but a more frequent and controlled power cycle of servers with HDD can make sure not a lot of servers go down in one go, when events like these happen. Also, gives you chance to identify patterns...

A lot more companies will go to a multi-cloud active active architecture with maybe even bare metal redundancies.

<i>&quot;As is often the case with a loss of power, there may be some hardware that is not recoverable...&quot;</i><p>No. Not even rarely. If they lost hardware because of this something much different than just loss of power happened on their servers&#x27; mains rails.