My personal take on the land of cyber security frameworks - and especially security standards - is that a good security team should be able to read through a list of controls (e.g. those in NIST 800-171) and express a reasoned opinion on each one with respect to the company's security posture. They are fantastic tools for reminding you what things you might have overlooked and driving a discussion about how your organization is approaching security - basically regardless of what type of company you run.<p>That's where the value stops - once you give lawyers, policymakers, and insurance companies access to these documents it becomes an unending game of regulatory capture, responsibility derogation, and box-ticking.<p>You end up with people who have zero context for technology running around demanding to see evidence that your smart toaster implements 12.2.14.1.5b "The centralized time server must enforce separation of duties" before it can be added to the network or some other such incoherent nonsense.<p>These standards always start in the right place, but they get used in the most frustrating ways because people who don't understand how technology works are, invariably, the auditors and assessors who apply these standards since true technologists can easily find more gratifying jobs doing literally anything else.
NIST CSF = The encyclopedia which breaks security down into as many areas/steps/sections as possible. If you are planning a 500-person security department, this is how you give them all something to do. The idea is to accomplish the task with manpower rather than elegance. CSF itself is mostly just a pointer to NIST 800-53. For truly large-scale operations it can be an ok fit, but for most organizations it is overkill that your cyber-insurance vendor will still expect you to do. Otherwise, best used as a reference not a guide.<p>ISO 27001 = Not super familiar with this one.<p>COBIT = The management/process focused version of NIST CSF. Great if you have an executive suite CTO, CISO, CRiskO, CPrivacyO, and want to coordinate their efforts in a program that divides responsibilities among them and associated committees. Includes maturity modeling, which gives it a +1, but it is distant from anything technology related. Instead, it is all about which committees should be formed to decide on risk management strategies etc...<p>PCI-DSS = You'll do this one because VISA makes you do it. Much more actionable than NIST or COBIT, but it depends on the third-party auditor who is issuing your attestation of compliance. "Your label maker has it's default password?" = audit finding.<p>CIS18 Controls = The most actionable/lightweight framework now that they have incorporated maturity levels (aka implementation groups). Not as thorough as NIST or COBIT. Well implemented, CIS18 is enough for most organizations provided they do not have a specific security standard or requirement in their industry.
We can tell how good these cybersecurity frameworks are by seeing how hard it is to breach the organizations certified at the highest levels such as SolarWinds [1], Equifax [2], Trend Micro [3][4], Cisco [5], and so many more.<p>It is so utterly ridiculous that anybody cares about these standards when literal clown shows get full marks.<p>The sign of a good standard is one that effectively and accurately predicts outcomes. A standard can be validated experimentally by evaluating if the certified targets conform to the stated predictions of the standard. A standard that fails to discriminate between good and bad is useless and the results of such certifications can be safely ignored.<p>The vast majority of cybersecurity frameworks have failed in these respects. Basically, if a standard gives Microsoft top marks, it is a lousy standard. That is not a necessary condition, but it is certainly sufficient along with anything certifying plenty of other security messes.<p>[1] <a href="https://www.schellman.com/certificate-directory?certificateNumber=1985896-2" rel="nofollow noreferrer">https://www.schellman.com/certificate-directory?certificateN...</a><p>[2] <a href="https://www.oxebridge.com/emma/equifax-held-iso-27001-certification-at-time-of-massive-system-hack/" rel="nofollow noreferrer">https://www.oxebridge.com/emma/equifax-held-iso-27001-certif...</a><p>[3] <a href="https://www.bleepingcomputer.com/news/security/trend-micro-fixes-bug-chinese-hackers-exploited-for-espionage/amp/" rel="nofollow noreferrer">https://www.bleepingcomputer.com/news/security/trend-micro-f...</a><p>[4] <a href="https://www.trendmicro.com/en_us/about/trust-center/compliance.html" rel="nofollow noreferrer">https://www.trendmicro.com/en_us/about/trust-center/complian...</a><p>[5] <a href="https://blogs.cisco.com/tag/iso-27001-certification" rel="nofollow noreferrer">https://blogs.cisco.com/tag/iso-27001-certification</a>
Being compliant within any of those frameworks does not make an organization secure. It's a good place to start, and will make the auditors happy, but assuming that (compliance equals secure) is a huge mistake.
The fundamentals of these frameworks are not inherently bad. The main problem is that they are treated as the checkbox solution that should be implemented regardless of context.<p>If you take your IT security work seriously you will in most cases already fulfill the requirements stipulated in the frameworks, and if you are setting out to get started with more structured approach they are a good place to start.<p>What cannot be understated however is the need for context, IT security is a set of choices that determine where you will have exposure and those business risk. For any business these will be different, as regulation and value creation differs. Exactly as any other risk management, but for some reason this does seem difficult for most to understand.
Having worked performing assessments against these frameworks and standards. The important thing people need to know particularly for NIST, is that you could have a clean bill of health today. But if one or two of your controls fails tomorrow, then you may run into trouble.<p>NIST isn't a maturity framework, it doesn't tell you that you need x control in place to be x level of maturity. It gives you control objectives that you need to design and operate effectively over time.<p>You might have a great set of controls across penetration testing and vulnerability management today, but if you fail to perform one of these controls tomorrow you will be vulnerable.<p>For example, I have performed controls assessments against organisations that have implement CIS controls. I have seen more often than not, CIS controls not being fully implemented as per the wording of the control AND the control not having been performed appropriately.<p>This is where using a cyber security standard and obtaining controls assurance is very important.
Also BSIMM and SAMM<p><a href="https://owaspsamm.org/blog/2020/10/29/comparing-bsimm-and-samm/" rel="nofollow noreferrer">https://owaspsamm.org/blog/2020/10/29/comparing-bsimm-and-sa...</a>