The Largest IT Outage in History
CrowdStrike just caused the largest IT outage in history. An update to the data their software uses to identify threats caused the Windows kernel to crash. While Microsoft may need to address some serious design issues, let's focus on CrowdStrike.
The file that caused the incident was entirely full of null characters. I don't know what CrowdStrike's testing and validation process looks like, but I do know that most enterprises can make this mistake without proper verification. Let's take a look at some compliance documents to see what we are REQUIRED to do.
NIST 800-53 SA-11
According to NIST 800-53 SA-11, both moderate and high baselines require:
- Developing and implementing a plan for ongoing security and privacy assessments.
- Performing unit, integration, system, and regression testing/evaluation at an organization-defined frequency.
- Producing evidence of the execution of the assessment plan and the results of the testing.
- Implementing a verifiable flaw remediation process and correcting flaws identified during testing and evaluation.
If compliant, CrowdStrike should have had a comprehensive testing and evaluation plan and evidence of its execution.
Implementing the Verification Process
Ensuring compliance and avoiding such catastrophic failures require stringent verification processes. Verification processes ensure that developers do not bypass testing protocols. This is where frameworks like in-toto and guidelines from NIST 800-204D come into play.
The in-toto Framework
in-toto provides a mechanism to secure the software supply chain, ensuring that every step of the software development process is verified. This means that every step in the SDLC is tracked and verified, ensuring that no unauthorized changes are made.
NIST 800-204D Guidelines
This publication outlines strategies for integrating software supply chain security into DevSecOps CI/CD pipelines. It emphasizes the importance of securing the entire software supply chain (SSC) by integrating security assurance measures into CI/CD pipelines. This framework provides actionable measures to enhance the security of cloud-native applications by addressing threats from both malicious actors and due diligence lapses.
The CrowdStrike incident underscores the importance of rigorous testing and verification processes. By adhering to frameworks like NIST 800-53 and integrating security measures outlined in NIST 800-204D and the in-toto framework, companies can significantly reduce the risk of outages caused by shipping the wrong, or improperly tested software.
We are hosting a webinar on in-toto next week where we are going to be talking about this. You can register here: Attestations in Action: Enhancing CI/CD Pipelines with in-toto
