Microsoft had three staff at Australian data centre campus when Azure went out

I guess people really have forgotten how to run datacenters.<p>The only people who should be shocked in this thread are the people who have been hoodwinked into thinking operations is so hard you need thousands of staff. I know AWS&#x2F;GCP&#x2F;Azure like to <i>charge us</i> as if we were hiring an army of sysadmins, but the truth is that day-to-day DC ops does not require so many people. Hardware failures are more rare than you think and you can work around them without panicking anyway.

Three to five on-site staff to operate a mid-sized DC (10MW&#x2F;~1K rack yield) is not unusual. This is assuming there are several others on-call.

I know very little on the datacenter operations side of things - I guess 3 people is not a lot, but what is normal? How many operations people are at say AWS US-East-1? I presume it doesn&#x27;t scale with number of servers, that would not scale well. What is a &#x27;normal&#x27; level? 10? 100? It can&#x27;t be more than 100, can it?

Guessing affected customers had to spend time and effort on top of ongoing high cloud bills<p>I&#x27;ve slept so much better since I began hosting, producing energy, and cooling on-prem

I wonder what a 4th, 5th or 10th person onsite could have done to speed up mitigation and recovery.

I straight out think MS lied when they said this was an .au only issue. We had a surge of rouge traffic from MS during this issue and we are pretty much on the other side of the globe.

It’s Microsoft. I’m kinda surprised it’s not ChatGPT-REPL at this point.