French here, and working for another french CSP. We lived the OVH incident live and saw the whole aftermath.<p>OVH was held liable because of the data loss, not for the service interruption. Data loss is something irremediable, permanent, definitive. Some businesses were basically ruined from this incident because they had no more data to operate. To add insult to injury, they sold offsite backups in the datacenter literally meters away. A service interruption, well, shit happens, and this is handled by SLA contracts that both parties agree to. You don't ruin a business (read: close a company) for a few days of outage.<p>I doubt CrowdStrike will be held liable for much; from corporations at least. They cannot repay the damage done, or they close the door. The healthcare sector is another beast, but I think it will come to more regulations for critical entities.
This headline is kind of misleading. It's actually someone's personal (educated) opinion on a blog, not a statement of fact. Should be something more like "I think CrowdStrike will be liable" or "CrowdStrike should be liable"
It's good to remind people that general liability waivers you often find with license agreements have no meaning outside of US jurisdiction if you're doing business in another jurisdiction.
Surely, there must be a gigantic number of claimants already taking to their lawyers about how to get compensation? Not just in France but across the planet?<p>I wonder how this kind of thing is organised, since there's all these jurisdictions.
I'm not a lawyer, and I'm definitely not a French lawyer, but I don't think the OVH comparison is valid.<p>In the OVH case, their backup system (as a whole) failed. Many customers were left with 0 data, and per the article "the court ruled the OVH backup service was not operated to a reasonable standard and failed at its purpose".<p>Meanwhile CrowdStrike "just" crashed their customer's kernels, for a duration of about 1 hour (during which they were 100% safe from cyber attacks!). Any remaining delays getting systems back online were (in my view) due to customers not having good enough disaster recovery plans. There's certainly grounds to argue that CrowdStrike's software was "not to a reasonable standard", but the first-order impacts (a software crash) are of a very different magnitude to permanently losing all data in a literal ball of fire (as in the OVH case).<p>Software crashes all the time. For better or for worse, we treat software bugs as an inevitability in most industries (there are exceptions, of course). While software bugs are the "fault" of the software vendor, the job of mitigating the impacts thereof lies with the people deploying it. The only thing that makes the CrowdStrike case newsworthy, compared to all the other software crashes that happen on a daily basis, is that CrowdStrike's many customers had inserted their software into many critical pathways.<p>CrowdStrike sells a playing card, and customers collectively built a house with them.<p>(P.S. Don't treat this as a defense of CrowdStrike. I think their software sucks and was developed sloppily. I think they <i>should</i> face consequences for their sloppiness, I just don't think they will, under current legal frameworks. At best, maybe people will vote with their wallets, going forwards.)
Can someone explain to me why the protections that Falcon provides, are not provided by the OS itself? I am not completely naive, I've secured quite a few critical Linux servers, but with Windows it seems that there do not exist the same clear roles of security. Contrast with Red Hat or even Canonical, where is feels like I'm (correctly) fighting the security of the systems to get them into a state where my users can use my applications.
I was aware of this being the case when dealing with consumers, but had assumed that because B2B contracts are assumed to be between 2 sophisticated parties that there is little legislative protection that could override the terms of the contract.<p>My understanding of law is generally UK based, but I'm not aware of legislation what would supersede a contract term limiting liability when the event that created the liability was one of general diligence/competence in carrying out the contract rather than relating to health and safety or some other area that is heavily legislated.<p>For that reason I'm unconvinced on the article's statement that this isn't just a "French Legal System" thing and that the same kind of judgement might be made in other jurisdictions.
not just in France<p>most(all?) EU have laws which limit how much you can opt out of liability _no matter what you write into a contract_<p>while I'm not sure about the exact boundaries per country but I'm pretty sure that at least all hospitals, emergency call services etc. can sue for a non-negligible part of the damages that outage caused directly<p>private people which where harmed by not getting operations done in time most likely can also sue them for the full damages caused to them (through it's hard to assess the damages and it might need to be indirectly by suing the hospital and the hospital sues for more damages)<p>what you likely will not be able to sue for is the lost opportunity cost, the man power needed to fix it etc.<p>also my guess is that for a lot of cases which are not as sever as human damages or as indirect as lost opportunity cost a huge factor will depend on the degree of negligence judges believe happened. And here "negligence" isn't limited to the specific change which caused the bug but also if they kept they due diligence in choices of tooling, approaches, business processes etc. to reasonable minimize the risk. (like e.g. was their way of parsing configs inadequate/did it follow industry best practices (IMHO it doesn't seem so), or was it adequate to mark the driver as required to allow boot (else windows would have auto disabled it and then restarted) etc.)
> "It is not an isolated incident. The same thing happened few weeks earlier with the CrowdStrike agent on Linux, nuking the system and there may be other occurrences before."<p>Is there a link with this incident?
I'm actually surprised the damage value I'm hearing about is not even $10B , I guess most of the downtime was on the weekend, but such a large scale 1-3 business day outage I'd think would a lot more. or perhaps it is because most small and medium businesses don't have crowdstrike because it is too expensive and they were not affected. Or another reason might be, indirect losses like the impact of delayed flights on individuals is not being considered.<p>I think if the total liability for Crowdstrike is less than a few years worth of revenue, they'll come out unscathed because as I understand, they are still not profitable, their valuation is purely on speculation on future revenue. Their biggest paying customers still care a lot about getting compromised, it isn't just a box checking exercise like many have suggested.
How deep does liability of a electricity provider go when they have major power outage? even if due to gross neglicence? would they be liable for all downstream failures including loss of life?
Sorry, but I feel the author is reaching for a conclusion.<p>From OP, in the OVH-case liability seems to override the contract / waivers when OVH was both the storage And backup provider and did not actively underline that this solution is suboptimal, in a situation where multiple data centers are physically very close. That's a chain of evidence.<p>For CrowdStrike, it is clear that the offering is to more mature counter parties (thus raising the B2B standard of evidence) and that CrowdStrike very essentially did not do / support staging, whatever. This is indeed bad industry practice, but one that can thought to be explicit from the start of the agreement. At least in my locale you either make explicit agreements OR industry standards are leading. We do not do industry standard X is pretty clear. Read the list in OP, replace CrowdStrike with Microsoft and then think of the international liability cases you've heard from where Microsoft was found liable for downtime, hacks and other issues.<p>Look, liabilities will always arise in such situations. But I expect only minor liabilities will arise. Mostly (AFAIK IANAL) the terms & conditions are applied in B2B-cases. This case is pretty obvious: you got what you signed up for. CrowdStrike with full scale access to your machines and no guarantees. On the other hand, Crowdstrike lost 125 billion in market cap. That's an indication of {liabilities + loss of future profits}. Pretty massive event for not being willing to do staging. But I expect it's mostly that CrowdStrike is tainted from now on. A friend of mine had a very bad stint as an employee of CrowdStrike recently and from what I learned from that case, I'm happy that the nature of the firm is somewhat more in the open now.
Another point against CrowdStrike: they did not have any "try once and if it fails, stop trying" logic. It cannot be the first time any CrowdStrike engineer saw the crash loop phenomenon. And so, a professional would have filed a high priority bug saying, "we need a way to stop crash loops definitively and automatically".<p>That would have been literally the headline I'd choose for the bug.<p>This is incompetence that in a just world would result in the corporate death penalty.
Holy shit (hits the fan).
For sure CrowdStrike will be held accountable in several countries, but I believe that some conclusions need to be drawn also from a customer/user perspective.<p>- Is it reasonable to grant such privilege access to a piece of software that ultimately is a black box ?<p>- Is it reasonable to put a Microsoft / Commercial / Closed source OS in critical infrastructure ? If not considered as critical, then “important” infrastructure ?<p>- Is it reasonable to have more than 70% of the computers/servers that run important infrastructure on the same OS / software ? How about the mitigation of the risks etc…<p>I sincerely hope that all of this CrowdStrike mayhem will push stakeholders to draw some conclusions and actions.
Why does article say "On 19th July 2019, CrowdStrike pushed an update" ?
Is it another incident in the past, same as OVH, or a typo? I'm kind of lost in context
Awesome. Falcon has been widely known (for years) as an utter piece of shit (code wise).<p>Maybe now ClownStrike will start testing it properly, hopefully thereby fixing the stability and other issues.
And yet there is no mention on the end-customers Change Management and Patch Management practices. Who pushes an update on 1000-5000-10000 machines without testing it?<p>To whoever does this I have only one quote from Jaws:<p>You go in the cage,
cage goes in the water,
you go in the water,
shark's in the water,
our shark.
Farewell and adieu to you, fair Spanish ladies. Farewell and adieu, you ladies of Spain.
> Does CrowdStrike do any testing whatsoever? Obviously they didn’t or the incident wouldn’t have happened.<p>Eh, parts of this article aren't very reasonable. Even if they did a buttload of testing, it only takes one failure in one part of the chain (near the end).<p>They didn't test something they should have, sure, but obviously they didn't do "no testing whatsoever"
What is hilarious to me is how the US government or courts doesn't seem to give a shit about this.<p>Corporativism in US is a thing. Companies can brick hospital systems killing patients, drive self-driving cars and run over people but don't get sued, and if they do, they settle for very little.<p>Just look at the recent Boeing incident where people were killed, the company clearly misled the US authorities and settled only a $0.5B fine.<p>Those companies in those scenarios should pay the fine that they should ($20B+), and if it means the company would go bankrupt, do it and form a new company diluting the previous shareholders.<p>Without doing this, shareholders and CEOs will have the incentive to carry on with their unfair practices that leads to dead people and deadlocked systems.