When I decided to move away from self-hosted git ages ago and then from Jenkins to GA two years ago, reliability was a huge factor in my decision because Github, I supposed, would be much better at keeping their infrastructure running than I am at keeping ours.<p>Turns out the uptime of both our git server and even the Jenkins instance beat GitHub by far and while the former only cost a marginal amount of CPU time on infrastructure I was running anyways, GitHub is a noticeable expense for us.<p>Of course it still saves me from the panic attacks every time I'm compelled to press the "Update now" button in Jenkins because either I do nothing and get my instance RCEd or I do press the button and who knows what plugin update will break which part of our setup, but while that was a constant fear in my mind, the amount of downtime caused by Jenkins plugin updates was zero whereas what GitHub is doing lately is way, way, way worse than zero.<p>I'm starting to get frustrated and like I presume many other paying users, I think I'm at a point where I feel like we should get partial refunds of our subscription money given the very spotty uptime all year now.
What's annoying about this is that the PR doesn't even say it's trying to run tests. It says everything is passing and just doesn't list the actions.<p>For a second I thought someone must have deleted the actions yaml files.<p>This is a dangerous failure mode.<p><a href="https://github.com/multiprocessio/dsq/pull/82" rel="nofollow">https://github.com/multiprocessio/dsq/pull/82</a><p>Screenshot here: <a href="https://twitter.com/phil_eaton/status/1542168020516216832" rel="nofollow">https://twitter.com/phil_eaton/status/1542168020516216832</a>
It seems a little early to call it "now resolved." I'm still seeing issues. If it's gotta munch through a queue or something, it would be helpful to announce that info.<p>Edit: It's just the HN title that says "now resolved." This github status says:<p>> <i>We have identified the source of disruptions and are actively working on a mitigation. The systems are in recovery and services are returning to green.</i>
I have been so happy to be on GitLab again after some time working at a company that used GitHub. The issues and epics in particular are so much better, and CI seems to be more reliable.
So, it seems like there is at least once a week a partial outage on github. For how long ate the CTOs and Engineering managers all around continue to accept this?<p>Who shouldn't <i>at the very least</i> donate a bit to the various open source CI solutions, as a way to have some kind of hedge?
Highly recommend <a href="https://builds.sr.ht" rel="nofollow">https://builds.sr.ht</a>. I've been using for ~2 years, never had an issue.
Again? Last time that happened was 9 days ago. [0] Just like I said before, at least twice a month GitHub Actions, Pages or something else goes down.<p>Each time this happens, it makes no sense to go all in on GitHub. Perhaps companies like ARM, and projects like Wine [1], ReactOS [2], etc already went with self-hosting or have a failsafe solution to fall back on.<p>[0] <a href="https://news.ycombinator.com/item?id=31815918" rel="nofollow">https://news.ycombinator.com/item?id=31815918</a><p>[1] <a href="https://www.phoronix.com/scan.php?page=news_item&px=Wine-GitLab-Main-Workflow" rel="nofollow">https://www.phoronix.com/scan.php?page=news_item&px=Wine-Git...</a><p>[2] <a href="https://github.com/reactos/reactos#code-mirrors" rel="nofollow">https://github.com/reactos/reactos#code-mirrors</a>
I think it is not just github actions, github in general is experiencing degraded performance [1].<p>[1] <a href="https://www.githubstatus.com/" rel="nofollow">https://www.githubstatus.com/</a>
Emm, sounds not good... this kind of incidents can impact not just developers but also business insight applications being heavily analyzing github activities and projects in real-time, like this one (<a href="https://ossinsight.io/" rel="nofollow">https://ossinsight.io/</a>).<p>Anyway to minimize the impact of such github incident on everyone's daily projects and business?
I don't really use github actions (like, ever), but it's the default setup for a terraform provider I'm working on and the very minute I queue a bunch of jobs; the system goes down. Interesting.
The company I currently work for is in the process of migrating out of Jenkins to Github Actions. With all problems that Github has, it has been, by far, a much better experience, even with all these issues and trade-offs, Github Actions in combination with the Github UI has been a net positive in all aspects.<p>Jenkins is slow and a nightmare to maintain. It became a huge ball of mud that nobody wants to touch. Just keeping the lights on it's a large burden for the infrastructure team.