Twitter was down

In case people are wondering what this is about: All links and images are not working on twitter right now. Seems like they accidentally included their own services in the api blocklist? Sending best of luck to the engineers. The whole twitter saga has given me a good chunk of valuable "lesson learned" over the last weeks and months.. so thanks for that.

Worth noting Twitter’s status page claims it is “operational” and no events are listed under history section for past 180-days, which is obviously wrong:<p>- <a href="https:&#x2F;&#x2F;status.twitter.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;status.twitter.com&#x2F;</a><p>* Not to mention the SSL certificate appears to be expired too.<p>———-<p>EDIT-01: Appears there’s a status page just for the API:<p>- <a href="https:&#x2F;&#x2F;api.twitterstat.us&#x2F;" rel="nofollow">https:&#x2F;&#x2F;api.twitterstat.us&#x2F;</a><p>EDIT-02: Wow, Twitter is using Twitter to post status updates:<p>- <a href="https:&#x2F;&#x2F;twitter.com&#x2F;TwitterSupport&#x2F;status&#x2F;1632792942262747136" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;TwitterSupport&#x2F;status&#x2F;163279294226274713...</a>

Twitter Support just posted:<p>&quot;Some parts of Twitter may not be working as expected right now. We made an internal change that had some unintended consequences. We’re working on this now and will share an update when it’s fixed.&quot;<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;TwitterSupport&#x2F;status&#x2F;1632792942262747136?s=20" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;TwitterSupport&#x2F;status&#x2F;163279294226274713...</a>

This is actually funny. Just this weekend I was talking to another engineer friend of mine, and we were taking bets as to how long it would be before twitter went down for the last time.<p>We felt bad for the engineers that are still there, and were actually surprised it&#x27;s stayed up this long, with all the knowledge that walked out the door. Basic stuff like &quot;where are our servers actually located&quot;.<p>I guess the folks that left loved Twitter enough to leave behind good notes, because even the most well architected systems still need some institutional knowledge when they go down.

Their website is broken for normal users. Try visiting anyone&#x27;s profile page over the web — you&#x27;ll get a 467 error<p>[0] e.g. Elon: <a href="https:&#x2F;&#x2F;twitter.com&#x2F;elonmusk" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;elonmusk</a>

“I would expect to start seeing significant public-facing problems with the technology within six months,” he says. “And I feel like that’s a generous estimate.”<p><a href="https:&#x2F;&#x2F;www.technologyreview.com&#x2F;2022&#x2F;11&#x2F;08&#x2F;1062886&#x2F;heres-how-a-twitter-engineer-says-it-will-break-in-the-coming-weeks&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.technologyreview.com&#x2F;2022&#x2F;11&#x2F;08&#x2F;1062886&#x2F;heres-ho...</a>

Speculation is that t.co link shortener service is broke :<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;emollick&#x2F;status&#x2F;1632788892599586816" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;emollick&#x2F;status&#x2F;1632788892599586816</a><p>&gt; The Twitter outage seems to be around a critical piece of infrastructure: the Twitter link shortener.<p>&gt; Every link on Twitter has been shortened to a t.co link. Only Twitter’s systems know where the link goes<p>&gt; If these systems ever go down, its link rot at scale

{&quot;errors&quot;:[{&quot;message&quot;:&quot;Your current API plan does not include access to this endpoint, please see <a href="https:&#x2F;&#x2F;developer.twitter.com&#x2F;en&#x2F;docs&#x2F;twitter-api" rel="nofollow">https:&#x2F;&#x2F;developer.twitter.com&#x2F;en&#x2F;docs&#x2F;twitter-api</a> for more information&quot;,&quot;code&quot;:467}]}

I can&#x27;t remember a major service nosediving so monumentally in such a short period of time.<p>Just a steady degradation of service.

It&#x27;s kind of funny that another thread (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35043330" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35043330</a>) is in top 5 Google results for &quot;Your current API plan does not include access to this endpoint&quot; search term, and is flooded with newly created accounts asking for support.

Even the homepage seems to be down as of 11:15am CST. Same error message.<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;</a>

I don&#x27;t think I could have predicted - even given the wanton layoffs - that the service would deteriorate so quickly across the board.<p>I barely use the app anymore and half the time stuff isn&#x27;t loading at all or is broken in obvious, overt ways.<p>Just ... stunning.

The Twitter suicide is the most crazy thing I’ve seen in my career. Rich and bored billionaire bought a “socially valuable service” just to drop its revenue by half and destroy it completely

The fun part is that the CEO is on Twitter as this happens, arguing about COVID.

Shameless plug: I created a browser extension to help transition to Mastodon[0]. If you don&#x27;t yet feel like you can leave twitter.com, but want to explore alternatives it&#x27;s a great way to get started. Essentially it injects Mastodon posts into your Twitter timeline, so you can retain your existing Twitter following while getting exposed to Mastodon.<p>[0] - <a href="https:&#x2F;&#x2F;chirper.picheta.me" rel="nofollow">https:&#x2F;&#x2F;chirper.picheta.me</a>

For context, most of Twitter appears to be having issues (t.co links and images are also broken)<p>Given all the random things Elon has cut from Twitter, it wouldn&#x27;t surprise me if that included a staging environment.

Not just Tweetdeck, all images and videos on the website too, and even the homepage doesn&#x27;t load.

If you like graphs, I&#x27;ve been watching this outage from OnlineOrNot (<a href="https:&#x2F;&#x2F;twitter.onlineornot.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;twitter.onlineornot.com&#x2F;</a>)

please see <a href="https:&#x2F;&#x2F;developer.twitter.com&#x2F;en&#x2F;docs&#x2F;twitter-api" rel="nofollow">https:&#x2F;&#x2F;developer.twitter.com&#x2F;en&#x2F;docs&#x2F;twitter-api</a> for more information

On an unrelated point, Californian employees in the main layoff were employed till January 4 for legal reasons. Many of them were demanding their contracted 2 months&#x27; severance to follow that. Now that 2 months have just passed, does anyone know if they received this pay?

<a href="https:&#x2F;&#x2F;status.twitterstat.us&#x2F;" rel="nofollow">https:&#x2F;&#x2F;status.twitterstat.us&#x2F;</a> says everything is operational

There were many different hypotheses on the outcome of Elon firing 80%+ of Twitter employees:<p>1. The site would crash and burn the next day.<p>2. Nothing would change. All those engineers were anyways just sitting around.<p>3. There would be no immediate impact (servers can run by themselves after all), but the site would slowly degrade over time as institutional knowledge around maintenance, upkeep and all the various system quirks was gone.<p>We are now seeing #3 play out in front of us.

Twitter in the age of Elon continues to lurch from one embarrassment to the next.

Twitter is broken for all users. Dang or someone else should merge this with the other twitter post on the front-page.

PSA: Mastodon, Misskey, Nostr, Bluesky exists now.<p>Just top 3 Mastodon instances(mastodon.social, pawoo.net, mstdn.jp) collectively has &gt; 2mil users, same level as Twitter in 2008. Misskey, an ActivityPub&#x2F;Mastodon compatible server software, now has &gt;10K users. Nostr isn&#x27;t much, Bluesky is still invite only and had just went past 2k registered, but those are all available options for direct substitutes to Twitter in case anyone still in need of a legacy twitterform social media.

It does appear that the undocumented api that nitter uses still works and is unblocked by this change as I&#x27;ve been able to access twitter through nitter.

If you like Tweetdeck and Reddit, checkout rdddeck [0].<p>[0] <a href="https:&#x2F;&#x2F;rdddeck.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;rdddeck.com&#x2F;</a>

The headline may overstate whatever happened.<p>I&#x27;d never used tweetdeck prior to now. Yet, it just now loaded fine for me. Once it loaded, I clicked a button that said something like, &quot;setup some fields&quot;, and then tweets were displayed. What&#x27;s not working? I&#x27;m just a free user of twitter.

Hardcore engineering at work.

&quot;This one API user has more calls than all the others combined.&quot;<p>&quot;Disable them and see what happens.&quot;

It feels like we&#x27;re being deliberately herded into Mastodon.<p>Obviously, that&#x27;s paranoid thinking. But the results are the same in many cases.<p>A market for lemons developing. A reinforcement of the strategy for others. And more stressful things in our lives.<p>If people are resentful, I understand. I hope they can find peaceful communities and safety wherever they go. I&#x27;m sure there will be political or technical reforms to accomplish that.<p>I hope people remember in the end that there were beautiful and good people in different spaces on the Internet, including Twitter.<p>I hope I can choose my own story.

I was literally adding a 2FA authenticator app to my Twitter accounts that aren&#x27;t Twitter Blue accounts when this happened. I thought it was something I did.<p>Good to know that I can rely on H&#x2F;N to tell me what&#x27;s really going on... and it&#x27;s not &quot;just me&quot;.

Twitter went down constantly in its early years and did fine; as long as errors get fixed, I don&#x27;t see why that dynamic would spell doom for Twitter now. There are a lot of people for whom the idea that Twitter can get along without most of its former headcount is a bitter pill to swallow, but that doesn&#x27;t make it untrue.<p>One reason the site was stable pre-Musk is that no one made any changes to it. For better or worse, it&#x27;s now owned by someone who actually uses the product and has ideas for how to develop it.

Their status page once again shows no issues. <a href="https:&#x2F;&#x2F;api.twitterstat.us&#x2F;" rel="nofollow">https:&#x2F;&#x2F;api.twitterstat.us&#x2F;</a>

Seems to be affecting web view but not mobile clients. Wonder if this has anything to do with them not paying their AWS bill as reported the other day.

I&#x27;m also getting the same error for the homepage.

Recursive error message, best error message.

This is all links. Not just tweetdeck.

It&#x27;s not just tweetdeck. Any clickable t.co URLs (i.e. all URLs embedded in tweets) are broken.

<a href="https:&#x2F;&#x2F;twitter.com" rel="nofollow">https:&#x2F;&#x2F;twitter.com</a><p>Is down too, at this moment.

<a href="https:&#x2F;&#x2F;downdetector.com&#x2F;status&#x2F;twitter&#x2F;" rel="nofollow">https:&#x2F;&#x2F;downdetector.com&#x2F;status&#x2F;twitter&#x2F;</a><p>Looks like it&#x27;s already being&#x2F;has been fixed. Still, pretty impressive level of brokenness for a short time.

Most Twitter subdomains are down with the same error.<p>help.twitter.com, api.twitter.com, developer.twitter.com etc.

They definitely need chaos engineering team. Maybe, they had one, but they were fired?

Even before the current downtime a string was data-mined from the TweetDeck beta which referred to itself as &quot;a feature of Twitter Blue&quot; so the days of being able to use TD for free are probably numbered anyway.

Does anyone know of time-series data with Twitter incidents&#x2F;up&#x2F;downtime? Would be interesting to do an analysis of the service level and change point analysis wrt the firing.

Yeah, they stopped paying for API access because it got insanely expensive.

This happened right as I was using Selenium to log in to Twitter for a project. Was wondering what the hell I was doing wrong until I saw this and realized it wasn&#x27;t just me.

Not that I particularly like Elon but people are now making a big deal out of every twitter outage as if twitter didn&#x27;t spend down half of the time before Elon bought it.

Surely this will be the end of twitter forever!<p>for reference: I see absolutely none of the things anybody is saying they are seeing as errors. Twitter works exactly as it ever has for me.

Tweetdeck provides access to cherry pick from a goldmine of fresh search data by &#x27;likes&#x27; and retweets. The latest tweetdeck is tough slogging to use.

Twitter devs don&#x27;t have Slack right now to even discuss this. They are all in one crowded conference call, panicking, and with Elon screaming.

Yep, something has gone very wrong. New images are also returning the same error. I assume the person that used to fix such things has been fired.

Twitter should not use a redirection service for links in the first place. Let a link be link. Ideally it should also not track click-throughs.

Incredible that MuskBird lacks the alien technology of checking the log to see which customers are still using the API before turning it off.

What&#x27;s up with that 467 error code? That&#x27;s not a standard HTTP status code; I wonder what it means?

2 full time bodyguards to protect Elon from employees when he goes go to bathroom<p>0 engineers to prevent this from happening

I am not able to access twitter in any way from my macbook&#x27;s browser.<p>I can access it from my mobile browser.

I get random random errors and can&#x27;t really browse (logged in)<p>Farewell?

The &quot;weird nerds defending Elon Musk&quot; meme gets more true as time goes on.<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;CaseyExplosion&#x2F;status&#x2F;1018558848670593024?lang=en" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;CaseyExplosion&#x2F;status&#x2F;101855884867059302...</a>

This isn&#x27;t &quot;you can&#x27;t run Twitter with half of its 2021 headcount&quot;.<p>This is &quot;Elon Musk is deliberately mandating bad software engineering practices&quot;. I will decline to speculate whether that is out of malice or incompetence.

@elon, what have you done?

No bio links work for me

It&#x27;s back

Y&#x27;all are silly and insufferable. Twitter had a temporary outage that was quickly rectified and you act like it’s some massive catastrophe directly caused by his highness Elon. Like you’ve never pushed a breaking change before…

Time to burn my imaginary internet credits again.<p>I won&#x27;t argue that firing all the engineers so swiftly wasn&#x27;t a mistake (I mean I _could_, but don&#x27;t want to dig myself any deeper than necessary in a single post). What I will say though, is that this gives me hints as to the quality of engineering that went on at Twitter.<p>I feel like some stellar Rube Goldberging went into solving problems that could have used simpler, if not _less CV boosting_, solutions. I&#x27;ve been at places like this. The engineers have fun building convoluted &quot;solutions&quot; for the sake of saying they worked on cool-tech-x. And surprise, it needs endless maintenance.