Last Friday was an unforgettable day for technicians and system managers around the world. The day started with a system outage that affected airports, financial institutions, hospitals, media outlets, supermarkets and offices around the world. A failure in the Falcon CrowdStrike antivirus update, which was executed automatically, caused the most widely used operating system, Windows, to crash, taking down thousands of computers.
On Friday, both CrowdStrike, the American cybersecurity firm responsible for the antivirus, and Microsoft published instructions so that users, mainly companies, could solve the problem. The solution was to delete the file containing the latest Falcon update and restart the computer.
Has it been resolved? Here’s what we know so far about the incident that reminded us of the terror of the “Blue Screen of Death,” which appears when Windows crashes.
How many people are affected?
Microsoft says 8.5 million computers worldwide have been affected, which is less than 1% of computers running the Windows operating system (which itself has a market share of more than 70%). These would be the Windows customers directly affected, since a single machine locked at an airport check-in counter, for example, could have affected many travelers at once. Microsoft has not provided any new details on this since Saturday, despite questions from this newspaper.
What happened specifically?
The first sign that something was wrong came from the airports. By early morning, the first flights had already been cancelled, which in the end amounted to 5,467 across the globe, and another 45,648 had been delayed. News soon began to arrive of failures in banks, electronic payment systems, hospitals and offices, among others.
It soon emerged that the problem was not the result of a cyberattack, but was linked to CrowdStrike, although there was no official confirmation from the company until almost noon. The latest update to Falcon, its most advanced antivirus, which monitors a series of threats in real time and using artificial intelligence (known in the jargon as EDR, endpoint detection and response), incorporated a problem in the code. That failure affected a driver (or device driver, a program that tells the operating system how to communicate with a hardware) which stopped Windows and caused the dreaded blue screen to appear.
The update was carried out in the early hours of Friday 19th (in Spain it was six in the morning, peninsular time), so it caught everyone off guard.
Has the problem been solved?
CrowdStrike pulled the problematic update shortly after the first incidents were reported. But that only stopped the problem from spreading further. At the same time, Microsoft Azure, the tech giant’s cloud computing division, released a guide to help its users restore their systems.
Windows users had to boot the system in Safe Mode, delete the last update file and restart the system normally. The problem is that this process had to be done machine by machine, and there are companies that have hundreds or thousands of them. In some cases, they also fell into a reboot loop, so you had to wait a few hours to complete the process. All this means that we still need days or weeks for some affected systems to recover. “It is said that, in some sectors, in less than a month it will be difficult to return to working as before,” says David Arroyo Guardeño, principal investigator of the Cybersecurity and Privacy Protection group at the CSIC.
Microsoft and CrowdStrike collaborated from the start, announcing that they were working on a patch for Microsoft Azure. On Sunday, it was ready: system administrators could download a recovery tool onto a USB stick that would automatically restore affected machines.
Could it have been avoided?
All experts consulted agree that Friday’s crisis was the result of an unfortunate chain of human errors. The latest update of the antivirus included a bug in the code, which someone wrote, but in addition, the relevant tests were not carried out before it was released. “The quality of the updates is key: they all have to be validated and thoroughly tested to ensure that they will not affect the system,” says Pedro Viana, head of pre-sales at Kaspersky, a competitor of CrowdStrike.
Another common practice in the sector, says Viana, is to divide updates into releases. “If they are sent gradually, the impact of failures can be less. If you see that there is a problem, you stop the rest of the releases and focus on resolving the incident,” he explains.
Who has been saved?
The CrowdStrike crisis spread to a large number of countries, from Australia, Thailand and India to France, Italy, Germany, the United Kingdom and Spain, as well as the United States and Mexico. The fall was global.
But there are two notable exceptions. Russia did not experience any problems. Russian Minister for Digital Development, Communications and Mass Media Maxut Shadayev said on Sunday that Moscow’s counter-sanctions measures against Russian companies had saved the country from a computer blackout. “The situation with Microsoft once again demonstrates the importance of import substitution of foreign software, primarily in critical information infrastructure facilities,” the minister said in remarks. collected by Business Insider.
China also had no major problems on Friday. In the Asian giant, Microsoft has a residual position: the big cloud dominators there are Alibaba, Tencent and Huawei. CrowdStrike, on the other hand, has hardly any presence in that market.
What risks are there now?
Hackers are experts at fishing in troubled waters. “From the moment Windows crashes, a gap can be created in which the machine is not protected,” says Viana. Kaspersky does not yet have data on how many security incidents the CrowdStrike update failure may have caused, but they have recorded several cases in which someone impersonates companies such as Microsoft itself to steal information from those who have suffered the problem.
Can it happen again?
No one is immune to errors. The first line of defense should be to establish protocols for reviewing and testing updates before they are released, something that large companies often follow strictly.
CrowdStrike is not the first such incident to occur, but it is the most far-reaching. In 2010, an update to McAfee’s antivirus software caused a problem with the Windows operating system. Customers were forced to reboot their systems. The company’s chief technology officer at the time was George Kurtz, now the CEO of CrowdStrike.
You can follow THE COUNTRY Technology in Facebook and X or sign up here to receive our weekly newsletter.
