GitHub was down again

I was seeing PRs failing to update & webhooks failing to trigger upon pushing code for 30 minutes before GH's status page acknowledged anything. I'm surprised they don't have monitoring in place that would catch webhooks failing within minutes of the failure beginning.

GitHub actions downtime is becoming painful for us. Having been lured on there with 10,000 included minutes which they shortly thereafter dropped to 3,000 I feel aggrieved paying for overages incurred from actions regularly shitting the bed.

Here we are again. Me taking a break on Hackernews because all my webhooks and pull requests are fucked and I have no idea where my devops tools are relative to what the real state of affairs is. I have pretty much had enough of this. It is too disruptive to our process. It is causing fragility and loss of confidence in our build pipeline.<p>At this point, we would probably be better off just bolting some lightweight git solution onto our devops tools (which are 100% custom in-house developed), rather than fighting with some more-durably-hosted offering of GitHub, et. al.<p>Anyone who posts that &quot;but you cant make it more reliable than microsoft&quot; line is not thinking about the dependencies between systems and the considerable impact incurred on a service just by virtue of it being a publicly-accessible platform without any cost barrier to entry. Sure, bringing it in house might bring additional difficulties, but I think I can eliminate a shitload of existing difficulties if we moved from webhooks across the public internet to a direct method invocation within the same binary image.

<a href="https:&#x2F;&#x2F;gitea.io&#x2F;en-us&#x2F;" rel="nofollow">https:&#x2F;&#x2F;gitea.io&#x2F;en-us&#x2F;</a><p><a href="https:&#x2F;&#x2F;git.zx2c4.com&#x2F;cgit&#x2F;" rel="nofollow">https:&#x2F;&#x2F;git.zx2c4.com&#x2F;cgit&#x2F;</a><p><a href="https:&#x2F;&#x2F;about.gitlab.com&#x2F;install&#x2F;?version=ce" rel="nofollow">https:&#x2F;&#x2F;about.gitlab.com&#x2F;install&#x2F;?version=ce</a>

There have been at least three major outages, e.g. git clone of a repo, in the past week alone. All three of which have been unreported (and NOT shown on their incident page), but I have email confirmation from GitHub support of these issues. It&#x27;s almost time to switch to Gitlab. I have hundreds of repositories, organizations, and packages to transfer, while it will be daunting... I need reliability. I have several paid GitHub orgs and accounts as well.

GitHub used to have a pretty cool status page, with all kinds of real time graphs. Does anyone know what happened to it? Since it makes me really sad that this status page is a plain lie, I had to visit HN to get the confirmation that they are having issues again, and that it just wasn&#x27;t only me.

The company I work for moved to Gitlab because we were pessimistic on GitHub in the past few years. I don’t really have a strong opinion on which is better though, I still keep my private repositories on GitHub. However, I feel that Microsoft will start feeling the pain soon as more people in the development community get sour on GitHub.

Why is outage history pre-acquisition removed from their history? If you try to go back in time it seems they only retain history up to a couple months after the acquisition. Is this just a 2 year retention policy or something being swept under the rug?

What&#x27;s the easiest way to duplicate all your Github repositories, with history, somewhere else?<p>Ideally, I&#x27;d like to have two synchronized repositories, for no single point of failure, organizational or otherwise.

Where does github publish post-moderms of downtime? I only see things like &quot;We have deployed a fix and are monitoring recovery.&quot; in the github status history which doesn&#x27;t provide details.

The Latest commits don&#x27;t appear in the commits tab

For free Gitea instance, You can try <a href="https:&#x2F;&#x2F;codeberg.org" rel="nofollow">https:&#x2F;&#x2F;codeberg.org</a>

Do I have to repeat this over and over again? If these non-profit open-source projects [0] are able to self-host a git solution like GitLab, Gitea, cgit or Phabricator instance somewhere, surely your team or open-source project can too.<p>Even a self-hosted GH Enterprise would suffice for some businesses but this would be overkill for others. I even see the Wireguard author using his own creation (cgit) to self-host on his own git solution for years. [1]<p>This is problematic since many JS&#x2F;TS, Go and Rust packages are on GitHub, which many developers rely on. Thus, it would be risky to think about tieing open-source project to (GitHub Actions, Apps, etc).<p>[0] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=23818020" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=23818020</a><p>[1] <a href="https:&#x2F;&#x2F;git.zx2c4.com" rel="nofollow">https:&#x2F;&#x2F;git.zx2c4.com</a>

That&#x27;s it, I&#x27;m launching SubversionHub.

How do they not have a single update in over 1.5 hours? This is ridiculous.

Ugh the last month has been pretty difficult. Hope they get better soon.

So that&#x27;s why my automated build wasn&#x27;t triggered ~4 hours ago. I was like &quot;no way github is having issues again, they were down just the other day, it&#x27;s probably just docker hub&#x27;s fault&quot;. If they decided to publish a blog post about these series of outages later, I bet it would be pretty interesting.

It&#x27;s been having issues all day. Wanted to show a coworker some changes I was proposing but the site wouldn&#x27;t show the changes I&#x27;d pushed to my pull request. Ended up just having him pull the changes.<p>FWIW the git backend always seems rock solid in comparison to the front end they have displaying it.

I really want to move to GitLab but its UI is atrocious... like too mobile phone looking on desktop

noticing issues on GitHub, CircleCI, and Launch Darkly.

I had a problem with github a while ago when I tried merging a PR to the master branch, the merge commits reflected on master but the PR was still open.I would repeatedly click the merge button but the PR wouldn&#x27;t show as merged

Likely unrelated, but I recently noticed that GitHub stopped updating my activities overview for july. I definitely pushed commits, but they are not noticed. Anyone else having a similar issue?

How is GitLab like in terms of downtime? I looked at their status history page and I&#x27;m seeing a lot of incidents but it&#x27;s hard to figure out what it actually means.

Running your own git server is trivial. I have been doing it for years on a very cheap digital ocean instance. Set up ssh keys, lock it down with ufw, done.<p>If that is not enough, run your own instance of gitlab.<p>If that is not enough use Gitlab.<p>Microsoft is going to attempt to make a profit on Github. That&#x27;s okay, but based on past experience and current issues, their business model is lock-in not service.<p>I suspect the same is true for NPM.

At the current rate, Whatever you host will have a better uptime than Github.

They&#x27;re probably using the Facebook SDK &#x2F;s

Scaling Rails is hard ? Github needs to move to CDN, static site deployment instead.

Git is distributed.

Did Microsoft adopt Scrum?

The Microsoft Effect

Don’t host yourself, it’s impossible to meet the reliability of the professionals