Tangential at best but I work in a regulated industry and we're starting to get some heavy scrutiny from external auditors and regulators on the way we plan to address extended outages and forced exits from third party vendors. This goes beyond basic DR/BCP and plans are reviewed with at least a high level scenario exercise.<p><a href="https://www.federalregister.gov/documents/2023/06/09/2023-12340/interagency-guidance-on-third-party-relationships-risk-management" rel="nofollow">https://www.federalregister.gov/documents/2023/06/09/2023-12...</a><p>On the surface a product like managed git repos would seem to be relatively straightforward to deal with, but these same regulated firms are also under tremendous scrutiny for access management, change management, SDLC, etc etc. They also have a huge software footprint which just increases the impact.<p>Self-hosting is obviously the traditional answer, but that's not exactly a simple one either.<p>Just an interesting problem.
It's almost as if only Git is distributed, but people sold out to Github for convenience. Too bad Git lacks a distributed bug tracker and wiki system like Fossil. Guess Github has to fail a lot more for things to change.
It was down for ~2 hours. The status website claims "degraded performance", but in reality we get<p><pre><code> git@github.com: Permission denied (publickey).
</code></pre>
Either GitHub didn't know how to communicate, or they were not sure about the real impact.<p>This is bad.
Experiencing this as well. I was a bit concerned something was wrong with my key but alas it is not my key but an outage.<p>Developer "snow day".
Not sure if I am just noticing GitHub's issues more often as I am using their tools pretty much every day but their availability is kinda not great. Be it Actions failing or something "not core business" (read: git operations) but I can't remember a month in the past where I was not annoyed by any outage on GitHub
Update: looks like the problem has been identified.<p>"We've identified a cause of degraded git operations, which may affect other GitHub services that rely upon git. We're working to remediate."
I guess it's a good time as any to setup a backup or check out alternatives that could mirror or standalone. Gitlab, Gitea, any others worth checking out?
Unclear if related, but the Terraform Registry is also having issues serving data: <a href="https://status.hashicorp.com/incidents/jypswvyh0h3z" rel="nofollow">https://status.hashicorp.com/incidents/jypswvyh0h3z</a>.<p>In a wonderful twist, we are relying on a couple modules served from GitHub!
Yup, SSH rejecting my key, thought it was something at my end but no. Can't run any Yocto builds... I guess a good opportunity to take a walk or do some life admin.
I've just ended my Pro plan; enough is enough. I've been debating moving my repositories to a self-hosted forge for a while; it'll be an interesting opportunity to move more CI operations out of Actions and into Nix.
> all Git operations were unavailable due to a configuration change<p><i>eats popcorn waiting for explanation of why they didn't catch it in non-prod</i>