The CrowdStrike Update: Testing Needs Evidence and Scope

On July 19, 2024, a CrowdStrike content configuration update caused widespread Windows system crashes. The initial version of this post was written that day. CrowdStrike’s later root-cause analysis describes the input-field mismatch and out-of-bounds read involved in the failure.

The earlier claim here that the cause was simply a file full of null characters was not an adequate account of the incident and has been removed.

Testing and its evidence

A test plan needs to define what will be exercised and what results count as acceptable. A record that a test ran is useful, but it cannot demonstrate coverage the test never had. A successful result is only as strong as the procedure and environment behind it.

NIST SP 800-53 SA-11 addresses developer testing and evaluation. Applicability depends on the selected requirements and system context; this article does not establish CrowdStrike’s compliance status.

Implementing the verification process

Bind a test result to the exact artifact or configuration being accepted. Identify the producer and retain the result and relevant test configuration. Require verification at the decision point, and make exceptions visible.

A gate can reject missing or unacceptable evidence. It cannot infer that an insufficient test would have caught an unknown defect. Staged rollout, monitoring, and recovery remain distinct controls.

The in-toto framework

in-toto helps represent and evaluate evidence about software supply-chain steps. It does not automatically observe every SDLC activity or prevent every unauthorized or defective change.

NIST SP 800-204D

NIST SP 800-204D discusses integrating supply-chain security into DevSecOps CI/CD pipelines. It provides useful context for provenance, attestations, and verification within a pipeline.

The lesson for a platform team is to make the release decision inspectable: the exact input, the checks performed, their results, the approving policy, and the limits of that decision. Explore the technical control workflow for how captured tests can support a review.