An update from Linode about the recent DDoS attacks

143 点作者 alexforster超过 9 年前

17 条评论

Glad to hear something official on this...5 or 6 days is way too long to go without something more than "We're working on it" and some light details. I understand that it's likely an all-hands-on-deck hair-on-fire situation over there, but those of us who rely on Linode for our own businesses have been largely left in the dark.When our customers are emailing and tweeting us and they just want to know when we are going to be up, and all we can say is "We have no idea, we don't know why this is happening or what's really going on", that's pretty much the definition of a worst case scenario from a customer service standpoint.As someone whose business relies on Linode currently to function, I am sympathetic to Linode's plight...this is the equivalent of someone coming and setting off a bomb in your factory; not exactly something that you can always plan for even if you have prevention measures in place. But they would have kept a lot more of my sympathy long-term if they would have communicated better with their customers in the first place...EDIT: And it looks like the attackers decided to start things back up again, as Linode.com is unavailable...

评论 #10820202 未加载

评论 #10820234 未加载

评论 #10820169 未加载

评论 #10820352 未加载

评论 #10820212 未加载

alexandrerond超过 9 年前

I can't believe people are criticising Linode:1 - Attack mitigation was mostly successful. As I thought and they have confirmed, the attack vectors evolved continuously.2 - They had to deal with this over Xmas. Anyone familiar with such a job knows what this means in terms of human resources, knowledge distribution, organization of technical response and communication with 3rd parties.3 - Linode is not Nagios. If you don't monitor your own infrastructure don't expect Linode to SMS you because your site might be down. Linode resources were focused on fighting the DDoS, as they should, and provided regular updates through their status site, as is expected. Everything else is nice-to-have, but no a must-have.4 - In line with what others said, I had 7 hours downtime in my London VPS. That is an uptime of 96% in the last 7 days. Considering restless DDoS ongoing over holidays, I'd say that is pretty good.I'm sorry, but what happens to Linode sucks, but it is an eventuality anyone with assets depending on this service should have counted with, because it can happen everywhere. Cannot blame Linode if your HA strategy does not exist, or you never thought of a way to gracefully fail over to a second provider if your business depends on >96% availability.

评论 #10820997 未加载

rebootthesystem超过 9 年前

My message to the attackers (in case they happen to read HN):Fuck you. I will continue to be a Linode customer. Not sure what your goals might be but you will not succeed.Frankly, and I am going to be politically incorrect here, these are the kinds of cases where I wish there was a "special forces" kind of task force to hunt down these pieces of shit and put them out of their misery.This amounts to financial terrorism of the worst kind. It affects small and large companies and creates untold losses across the board. It is entirely unproductive. The world would be a better place if the pieces of shit who engage in this sort of financial terrorism simply didn't exist.Happy New Year.Linode folks: I'm renting another server next week. Don't need it. Just want to support your effort and, in a tiny way, help mitigate losses. I might just give it to the kids in the robotic team I mentor so they can play around in a real server environment.

评论 #10820480 未加载

评论 #10820596 未加载

评论 #10820619 未加载

atom_enger超过 9 年前

Thanks for the update and the hard work. You all work a job that requires a lot of sweat and tears that goes unappreciated from many levels of our society. Making the internet work is hard work.Know that you're in good company here and that we're rooting for you.

评论 #10820492 未加载

评论 #10820543 未加载

jafingi超过 9 年前

My London Linodes' stats from the past 7 days: 40 outages. 6h25m downtime.What upsets me the most is that customers haven't heard a single word from Linode. If you weren't watching their status page (and have server monitoring on your servers), you'd be clueless.The least they could do is to email affected customers about what's happening, and the time frame they need to fix it.But one week of continuous issues is just not good enough.

评论 #10820585 未加载

scurvy超过 9 年前

1) Why in the world are you exposing your router control planes to the outside world? That should be ACL'd off (in stateless firewall rules and routing engine) to only allow access from a few IP's.2) Your transit providers should be defending their infrastructure. I've never seen a transit provider allow an attacker to take out their /30 serials or IX addresses. This is their network after all. If attackers try to hit the serial between customer and provider, you just readdress the serial to RFC1918 space. You don't really need a routable address there other than to make traceroutes easy to read. If they attack farther upstream in the provider's network, you just add ACL's at the provider edge. Nothing external will ever need to reach a provider's core. This is basic, basic stuff.Next time, don't only run your network on house bandwidth (HE, TelX, etc). Or in other words, caveat emptor.

评论 #10824710 未加载

pbowyer超过 9 年前

Link to original thread, which has been getting comments up until today: <a href="https://news.ycombinator.com/item?id=10806686" rel="nofollow">https://news.ycombinator.com/item?id=10806686</a>

bm98超过 9 年前

> a bad actor is purchasing large amounts of botnet capacity in an attempt to significantly damage Linode’s business.I wonder what size investment this is taking, and what the end-game is for the bad actor. Unless Linode's mitigation tactics are increasing the bad actor's costs, what's to stop the bad actor from continuing the attacks until Linode goes out of business?

评论 #10820975 未加载

评论 #10821012 未加载

psxhacker超过 9 年前

Greetings.I know I migth come late, but being one of your very satisfied Customers, and having experienced this type of issue multiple times with other providers that didn't even bothered to even ackowledge that there was an issue, I can say that I wil remain with you regardless.Also to those saying "I have all my business running at Linode so this is unacceptable", I only says this: You get what you pay for, and for a VPS Service you won't find better than Linode, and if you have something critical running for YOUR Clients, than it is YOUR responsability to ensure resiliency against this type of situation. Linode is a VPS provider after all, and the reason why you are making money out of someone who doesn't know enougth to go to the VPS hoster themselfes.Good luck making a profitable business and milking your Customers running on AWS or AZURE. You'd be broke and in debth at the first DDOS and Over-Bandwith charge from any of them.I work at a service provider myself, and I understand what you guys had to deal with the last 10 days, and you have my full support.

switch007超过 9 年前

I'm a bit surprised to see the update from someone who is not senior management/C-level (as far as I can tell). Where is the communication from the CEO/CTO?It seems a bit unfair to have this fall on Alex's shoulders. I could be way off base, happy to be put right. I'm sorry to hear about your ruined holidays. Hopefully you'll get some time off soon :)

tunesmith超过 9 年前

That's crazy... I have a linode box with several low-traffic websites on it, old projects I've wanted to keep around for archival purposes. I picked linode because I wanted root access and they were cheap but really just because I wasn't sure of better options. I suppose there is always t2.nano.

评论 #10820449 未加载

评论 #10820312 未加载

circuit_breaker超过 9 年前

Linode has always been a great host. Sure they've had their growing pains but I've never been more happy with a virtual hosting provider, even their support. But yes, days without communication is not a good thing. Let's hope they learn from this.

vox_mollis超过 9 年前

ISPs already spend plenty of money on DPI and HTTP injection gear. It would cost next to nothing to do basic egress filtering and detecting+throttling known compromised customers.And yet, we still get DDoS attacks. Why?

评论 #10823311 未加载

danieltillett超过 9 年前

I think the lesson here is don't rely on one supplier. I have my tiny infrastructure spread over three different suppliers in different geographical locations. Plan for the worst and hope for the best.Edit. This is in no way a criticism of linode. The worst outcome is if we all end up with one monopoly supplier. I have deliberately avoided using the big player in this space as I want support diversity. This makes my job harder, but it is better for us all if we don't put all our eggs in the one basket.

评论 #10820436 未加载

rast-a超过 9 年前

Out of curiosity: why is it so hard to track the real origin of DDoS attack, who's behind them and what they are after?

评论 #10820463 未加载

评论 #10820438 未加载

elinchrome超过 9 年前

How do botnets still exist? Wasn't that a problem that should have gone away with Windows XP?

评论 #10822601 未加载

评论 #10822232 未加载

geuis超过 9 年前

jsonip.com is hosted on Linode. It's been averaging roughly 6mb/s inbound for months, but in the last week it's been about 8.5. I'm not sure if the uptick has anything to do with the DDOS attacks or not.

评论 #10820604 未加载