GitHub is down

Hearts out to all the GitHub SREs who get to fix this, and all the SREs at GitHub’s customers who get to spend the next week convincing management and their peers that moving git in house won’t result in fewer outages.

The fact that GitHub being down can have such an impact means that Git’s decentralised features don’t go far enough. Why aren’t issues, PRs and all these other little things we depend on also stored in a git repo? Or something similar that can withstand GitHub being down.<p>I set aside time today to go through my notifications and I can’t because of this. Notifications are trickier to decentralise but still, ugh.

Mine looks like (on git push):<p><pre><code> remote: Resolving deltas: 100% (4&#x2F;4), completed with 4 local objects. remote: fatal error in commit_refs To $REPO ! [remote rejected] main -&gt; main (failure) error: failed to push some refs to &#x27;$REPO&#x27; </code></pre> Status Page: <a href="https:&#x2F;&#x2F;www.githubstatus.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.githubstatus.com&#x2F;</a> -- actually shows red -- it was almost updated as fast as HN.

Wonder why mine[1] got marked as a duplicate when it was posted first?<p>1. <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35817640" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35817640</a>

Everything is down. How can this happen? Many of these services are unrelated, like even the brand-new Codespaces and Copilot are completely down. Do they all have a single point of failure? (Presumably the git server)

&gt;This incident has been resolved. Posted 1 minute ago. May 04, 2023 - 16:23 UTC<p>Everything resolved at the same time. DNS?

I&#x27;m not really sure Github prioritizes the high availability anymore. These outages might be a way to motivate the customers switch to Github&#x27;s self-hosted enterprise solution which is the revenue stream for Github. If they&#x27;re not losing users because of these outages, (I&#x27;m sure they can find that out) these outages really make them more money in the long run..

Last time GitHub went down was less than a month ago: [0]<p>At this point maybe it is time to self-host rather than to continue to tolerate this since it&#x27;s guaranteed to goes down every month. [1]<p>Even open-source projects like RedoxOS, GNOME, KDE, ReactOS, etc are doing just fine without worrying about GitHub&#x27;s unreliability as predicted years ago on not centralizing everything to GitHub. [2]<p>[0] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35611739" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35611739</a><p>[1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35611862" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35611862</a><p>[2] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22867803" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22867803</a>

I sense a disturbance in the force...

Quick, activate the Copilot-based sysadmin!

Hmm, will it turn out to be DNS?

I wonder if they are migrating most of their platform to Azure.

Huh, it is working for me. Is it region specific?

27 minutes to resolve - relatively fast

According to the same status page, this is <i>Resolved</i> as of May 04, 2023 - 16:23 UTC.

Their twitter account states that allnis back to normal

I see GitHub is working hard to achieve nine fives.