Lessons from the crash

By James Jimenez

Most of us never even noticed that anything was wrong. And yet, across the world, airlines, banks, and television stations were brought to their knees by a software glitch. Starting late Thursday night, going into Friday, computers everywhere started crashing. By 3 AM, the US Federal Aviation Administration had announced the grounding of two major American airlines. Others soon followed, with reports of tech outages coming in from all over the world, including Spain, Australia, and the Philippines. Yes, local airlines and banks did experience technical issues leading to flight cancellations and causing banking services to go offline.

CrowdStrike, a US cybersecurity company, quickly owned up to the problem. Apparently, there had been a software update had caused its Endpoint Detection and Response (EDR) product to crash. EDRs, like the CrowdStrike’s Falcon Sensor, are smart security systems that are installed in computers, constantly monitoring the system, looking for unusual activities that might indicate a threat, such as a hacker trying to break in. In this case, a software update failed, causing the Falcon Sensor to crash, bringing the computers – or endpoints – to a screeching halt, effectively hobbling millions of devices globally, and displaying the dreaded Blue Screen of Death.

The lesson is clear to those willing to see it: They may be incredibly useful, but when a simple and innocuous thing like a software update failure could easily lead to widespread system malfunctions, then we need to acknowledge the inherent fragility of the digital infrastructures we have come to rely upon.

Tech in Elections

The problem has since been fixed, but the CrowdStrike incident dramatically underscored the inherent risks and dangers that come with increased reliance on digital systems. And with the COMELEC leaning heavily into technology to enhance the efficiency, accessibility, and transparency of Philippine elections, it should be seen also as highlighting the vulnerabilities that can compromise the integrity of elections and the democratic process.

Up to this point, COMELEC has been very careful to isolate the technology systems directly used for vote counting and canvassing from unauthorized access. The counting machines and canvassing systems are all offline except for the tiny window of time when election results are being transmitted out, using the strongest encryption protocols, to the next canvassing level.  However, with the Constitutional body’s recent push to use internet voting in 2025, with an eye to expanding its use even further in future elections, along with the increasing role of digital technology in election administration, such as for COMELEC’s planned remote voter registration, the vulnerabilities are rapidly stacking up.

Mitigating Vulnerabilities

The CrowdStrike outage of 2024 has been a pivotal event, shedding light on the vulnerabilities of our digital age and the critical importance of preparation, resilience, and continuous improvement. By learning from this incident and implementing the necessary measures, organizations can better safeguard against future disruptions, ensuring the stability and reliability of their technological systems. This is, to my mind, most true for the COMELEC as it is charged with ensuring that the people’s sovereignty – as exercised through suffrage – is not undermined in any way, least of all by a faulty software update.

Studying how the outage unfolded through the lens of elections and election administration, these are my takeaways as an avid democracy observer:

First, decentralizing election systems can reduce the impact of single point of failure, and enhance resilience against outages and attacks. In practical terms, this might mean having to rethink the centralized nature of the current system where all the data from the voting machines are collected and transmitted to a central server for consolidation and canvassing. While this arrangement streamlines the process and ensures the quick dissemination of results, it is also particularly vulnerable to cyberattacks and single point of failure scenarios.

Second, rigorous testing and quality assurance protocols for software updates and system components are crucial. Needless to say, this is what the automation law’s requirement for international certification addresses, but it should also be interpreted to be a warning against avoiding the use of untested software and applications for mission-critical systems. And more than just putting the automated election system through rigorous testing, COMELEC would also be well served by conducting regular “war games” that simulate potential failures. Taking the time to do these tests will help ensure that the contingency plans are robust and actionable.

Third, the hardening of vulnerable systems is essential. This will involve regular security audits, the use of threat detection systems, as well as ensuring that everyone involved in the election process is trained in cybersecurity best practices. This last one is often a blind spot for many government organizations where employees tend to dismiss cybersecurity necessities as something only “techie-techie” people should be expected to understand.  In reality however, even people who never see the inside of a server room can be the instruments for a cyberattack that ultimately incapacitates the entire system.

COMELEC must also invest in collaborative activities with cybersecurity experts and organizations. I have the highest confidence in the agency’s IT department, but outside experts will provide additional layers of protection and rapid response capabilities in the event of an attack or system failure.

Fourth, as COMELEC learned in the wake of the “seven-hour glitch” of 2019, maintaining transparency with the public is vital. Elections rise and fall on the strength of public perception, and clear communication about the steps being taken to secure election tech and the contingency plans in place can help build and maintain public trust. In the event of disruptions, timely and transparent updates are indispensable for mitigating panic and blunting the corrosive effects of misinformation.

And finally, the importance of continuous voter education cannot be overemphasized. Informing the public about the role of technology in elections, the steps taken to secure their votes, and what to expect during potential disruptions can foster a more informed and resilient electorate.