Business Continuity Management / Disaster Recovery , CrowdStrike Outage Updates , Endpoint Security
CrowdStrike's Response to Outage Will Minimize Lost Business
Technical Analysts Emphasize Need for Enhanced Security Testing, Quality AssuranceOne of the more extreme customer reactions to the CrowdStrike global IT outage came on Friday from Elon Musk. The Tesla founder complained on X that the outage had hit the company's suppliers. But rather than fix the faulty software update on Tesla's Windows machines, Musk said, "We just deleted CrowdStrike from all our systems, so no rollouts at all."
See Also: 2024 Threat Hunting Report: Insights to Outsmart Modern Adversaries
Experts and analysts aren't ready to write off CrowdStrike as Musk did, but they said the company must enhance its testing and validation procedures and address deficiencies in its current quality assurance processes to minimize customer and prospect attrition.
Analysts urged the Austin, Texas-based endpoint security behemoth to conduct a thorough technical retrospective to understand the root cause of the faulty CrowdStrike software content update and make necessary changes. Customers will expect both greater transparency as well as a more rigorous quality assurance and testing process to ensure their endpoint security tool doesn't cause a massive outage.
"When there's an issue, you need to be proactive, transparent and aggressive in your response," said Frank Dickson, who leads IDC's security and trust practice. "Any hesitancy in slowing either the flow of information or not owning up to it, you will pay for it. All proactiveness will pay dividends in the future. CrowdStrike, in terms of their response, I would put it in terms of almost the best. They've been exceptional," he said.
CrowdStrike leaders have been candid about the issue. Chief Security Officer Shawn Henry said, "The past two days have been the most challenging 48 hours for me over 12+ years. The confidence we built in drips over the years was lost in buckets within hours, and it was a gut punch … We let down the very people we committed to protect, and to say we're devastated is a huge understatement."
CrowdStrike Stock Dip Expected to Be Short-Lived
While technical analysts praised CrowdStrike for the speed and thoroughness of its response to Friday's outage that affected 8.5 million Windows hosts, financial analysts have been more cautious. Guggenheim and BTIG downgraded CrowdStrike's stock Monday, and Guggenheim analyst John DiFucci anticipating "resistance to new deals in the near-term" since "the restoration of its reputation may take more time" (see: Microsoft Sees 8.5M Systems Hit by Faulty CrowdStrike Update).
CrowdStrike's stock is down $77.74 - or 22.5% - to $267.36 per share since the outage occurred, which is the lowest the stock has traded since Jan. 8. In contrast, rival SentinelOne's stock is up $3.31 - or 14.8% - to $23.31 per share, which is the highest the stock has traded since March 8. A Baird analyst reported seeing "revived interest in specialized best-of-breed vendors to diversify from single-vendor dependency."
Forrester principal analyst Allie Mellen said she has heard chatter about customers implementing multiple EDR or XDR providers to mitigate risk but that it would be practically difficult due to complexity, resource requirements and added workloads for security teams. Some analysts expect lower win rates or longer deal cycles for the next six months, but they anticipate the sales impact will dissipate as 2025 nears.
Mellen said customers will likely want to implement their own enhanced controls but cautioned that the frequent nature of content updates - which are what caused Friday's outage - complicates versioning and makes thorough pre-deployment testing challenging. While larger enterprises might do more of their own quality assurance, most companies lack the resources for extensive testing before doing updates.
Forrester senior analyst Paddy Harrington said businesses might reconsider automatic software updates, meaning vendors may need to offer opt-in options for automatic updates. Gartner director analyst Eric Grenier said CrowdStrike’s acceptance of responsibility is critical to regaining trust, and detailed root-cause analysis and changes in QA procedures are essential for gaining future confidence (see: CrowdStrike Disruption Restoration Is Taking Time).
How Should Rivals Respond to CrowdStrike's Misfortune?
Responses from endpoint security rivals of CrowdStrike have ranged from empathy and support to more boorish behavior and aggressive sales tactics. Followers on LinkedIn praised Sophos CEO Joe Levy for writing, "We should also avoid 'we don't crash systems' claims while people's worlds are upside down" since "moments like this happen to the best of us, and we all deserve peer support."
But many panned Cybereason for launching an "emergency hotline" to help companies affected by the CrowdStrike outage, and clients were told to call 1-833 NO CROWD. The company has since removed the news release announcing this hotline. Trellix CEO Bryan Palma said what happened at CrowdStrike couldn't happen at Trellix since the company does phased updates and lets clients decide what to deploy and when.
Harrington expects businesses to push CrowdStrike and other software providers with kernel-level access to include vendor-induced outages in their warranties rather than just protection against security incidents. While the outage will slow new business acquisition, analysts don't expect mass customer switching due to high costs and CrowdStrike's overall trust within its client base.
Despite customers such as Musk, "switching is not an easy task," Dickson said, "and CrowdStrike has built up a lot of goodwill and trust within its customer base."
Going forward, analysts would like to see both enhanced quality assurance and testing procedures - possibly involving multiple labs or stages - as well as detailed documentation and adherence to those procedures. Customers should inquire about the specific QA processes, testing procedures and security measures that vendors have in place to ensure updates do not cause similar issues in the future, they said.
"This is going to be just a little blip for CrowdStrike," said IT-Harvest chief research analyst Richard Stiennon. "CrowdStrike is so well-respected, and its customers are so happy with the products, that they'll continue to be customers."