Oh good. The attackers were only in their systems since 9/4/19 before being detected on 12/12/20, so only 15 months of infiltration into SolarWinds' systems before detection. At least the payload was only deployed 2/20/20, so their customers were only completely infiltrated without detection for 8 months. Assuming the attackers could only get a 10 MB/s channel <i>total</i> per target even though they probably infected thousands to tens of thousands of machines per target, at ~20 million seconds that would constitute ~200 TB exfiltrated per customer or ~19 years of 1080p video.<p>If an attacker has just one day to root around and exfiltrate they can easily get valuable information. If they are given 8 months they have already gotten everything of value for months and are just waiting around for any new data to come in. Think how inadequate your systems must be to let an attacker sit around in your systems for 8 months, it is mind-boggling how unqualified their systems are for their problems. And this is not just an indictment of SolarWinds. Just in case anybody forgets, it was the top-flight security company FireEye who discovered this breach after realizing they themselves were breached. A "best of the best" security company took 8 months before realizing that they or any of their customers had been breached. This is what "top-class" security buys you.
This is the key excerpt, its quite shocking:<p><pre><code> Crowdstrike said Sunspot was written to be able to detect when it was installed on a SolarWinds developer system, and to lie in wait until specific Orion source code files were accessed by developers. This allowed the intruders to “replace source code files during the build process, before compilation,” Crowdstrike wrote.
The attackers also included safeguards to prevent the backdoor code lines from appearing in Orion software build logs, and checks to ensure that such tampering wouldn’t cause build errors.
“The design of SUNSPOT suggests [the malware] developers invested a lot of effort to ensure the code was properly inserted and remained undetected, and prioritized operational security to avoid revealing their presence in the build environment to SolarWinds developers,” CrowdStrike wrote.
</code></pre>
So how do we guard against this type of attack? How do we know this hasn't already happened to some of us? What is the potential fallout from this hack, it seems quite significant.<p>This must be why the Japanese Intelligence agencies prefer paper over computer systems. The digitization of critical national security apparatus is the Archilles Heel that is being exploited successfully. One example is Japan's intelligence gathering capabilities in East Asia, especially China, which is bar none. Japan has a better linguistic understanding of the Chinese language (Kanji and all) but also interestingly much of PRC's massive public surveillance equipment like CCTV cameras are made in Japan.<p>Even if they hire Krebs, I believe that if its digital, it can be hacked given long enough time period and unlimited state level backing and head hunting essentially geniuses of their country to do their bidding. I wonder how Biden-Harris administration will respond, it is very clear who the state actor is here. I'm very nervous about the ramifications of this hack.
As a citizen, I am shocked and appalled by this backdoor. As a software engineer, I can't help but marvel at the creativity and thoughtfulness put into the exploit.
Yes, but...<p>Having worked at SolarWinds they're especially susceptible to demands from sales and marketing. "Go faster, ignore tech best practices, etc." It's not unique, but their culture is not a dev-first, or security-first, culture to say the least. Many product managers answer to marketing first, and don't have earnest tech backgrounds that would let them know right from wrong past sales numbers. The culture changed significantly when they went public the first time; it went from a place where devs built good tools... to a place looking to buy products / competitor products so they could charge more for their services. Look at how long it took them to get into cloud tools -- great example of how marketing and sales missed the boat because they were only focused on things they had sold before and not focused on systemic changes to the industry -- because technologists weren't driving.<p>Anyway, like I've worked a lot places with better security built into the culture, better tech best practices built into the culture... that's all I'm trying to say. Knowing that attacks like this are out there... and it was just a matter of time before it happened, SolarWinds did next to nothing to avoid it happening to them.
<a href="https://www.crowdstrike.com/blog/sunspot-malware-technical-analysis/" rel="nofollow">https://www.crowdstrike.com/blog/sunspot-malware-technical-a...</a> is the key link with more technical analysis for those interested, including source code of the implant.<p><pre><code> "If the decryption of the parameters (target file path and replacement source code) is successful and if the MD5 checks pass, SUNSPOT proceeds with the replacement of the source file content. The original source file is copied with a .bk extension (e.g., InventoryManager.bk), to back up the original content. The backdoored source is written to the same filename, but with a .tmp extension (e.g., InventoryManager.tmp), before being moved using MoveFileEx to the original filename (InventoryManager.cs). After these steps, the source file backdoored with SUNBURST will then be compiled as part of the standard process."</code></pre>
I'm surprised this hasn't caused the software industry to completely halt and rewrite/audit all third party libraries and dependencies. The entire software supply chain is highly trust-based, npm especially. Why aren't we seeing the start of a NIH dark age?
I am 99% percent sure the hackers are among the CUSTOMERS of Solar Winds.<p>That way they were able to live-test infected SolarWinds distro in their own controlled environment and develop all possible mitigations and techniques - the sheer amount of these evading techniques suggests they were built up over time, and not instantly.<p>Being Solar Winds customer and receiving infected updated versions every time gave them opportunity to perfect their techniques and hide for so long<p>At least that what I would do if I were a hacker and wanted to persist and be very careful about not getting detected
The scope of this thing is staggering to me.<p>It must have taken a significant amount of time
in prep and dev and then deploy and control.<p>A serious investment in time and resources so money.<p>But why?<p>In order to burn so many nice tricks they had to be after
something quite valuable in one way or another.<p>What was the motherload they were after and did they get it?
Will we ever know?<p>Or was this harvesting of the intelligence and information there were needed for the real gold?<p>They went wide, which might have been to obscure the real
target, or they needed a lot of pieces from different sources
I wonder is something as simple as two factor authentication for those who have access to the build servers may have helped prevent something like this attack?
At one end of the spectrum I need to make sure people aren't choosing dumb passwords and are applying software updates. On the other end, centralizing control makes a very juicy target for hackers.
From what we've seen until now, this company deserves everything that happened to it. Hope they go under. Ignoring security best practices for the chance of a quick buck.