The latest version of the Falcon Sensor software was intended to update the threat defenses that CrowdStrike clients use to make their systems more secure from hacking. However, a faulty code in the update file resulted in one of the most widespread technology outages in recent years for companies that use Microsoft’s Windows operating system.
Global banks, airlines, hospitals, and government agencies were shut down. CrowdStrike has released information on how to fix the affected systems, but experts say it will take time to get the systems back online because the flawed code will have to be manually removed.
“It’s possible that this file was not included or was missed through the vetting or sandboxing they do when they look at the code,” said Steve Cobb, chief security officer at Security Scorecard, which says the issue affected some systems.
The issue quickly surfaced after the update was released on Friday, with users posting photos on social media of their computers showing blue screens displaying error messages — what’s known in the industry as the “blue screen of death.”
Patrick Wardle, a security researcher who specializes in studying threats to operating systems, said his analysis identified the code responsible for the outage. He said the issue with the update was “in a file that contained configuration information or signatures.” These signatures are code that detects specific types of malicious code, or malware. “It’s very common for security products to update signatures about once a day, because they’re constantly monitoring for new malware and trying to make sure their customers are protected from the latest threats,” he said.
“The frequency of updates may be why (CrowdStrike) hasn’t tested it as much,” he said.
It’s unclear how the flawed code ended up in the update, or why it wasn’t detected before it was released to customers.
“Ideally, this would have been released to a limited pool first,” said John Hammond, senior security researcher at Huntress Labs. “This is a safer approach to avoiding this kind of chaos.”
Other security companies have had similar incidents in the past, including McAfee’s buggy antivirus update in 2010 that caused hundreds of thousands of computers to crash.
But the global impact of the outage reflects CrowdStrike’s dominance: More than half of the Fortune 500 companies and many government agencies, including the Cybersecurity and Infrastructure Security Agency, the nation’s top cybersecurity agency, use the company’s software.
