AI’s Real Danger May Be Digital Chaos, Not the End of Humanity

A critical flaw in PHP, software used across much of the internet, remained difficult to exploit for years until AI helped researchers turn it into a practical attack. The episode illustrates a more immediate AI threat: not necessarily machines destroying humanity, but technology enabling disruption on a scale that businesses, governments, and essential services may struggle to contain.

Predictions about artificial intelligence often swing between two extremes: a future in which machines solve humanity’s greatest problems, and one in which they become so powerful (and possibly independently autonomous) that they threaten our very existence. Both possibilities attract plenty of attention, but there is another, more immediate concern that deserves at least as much scrutiny.

Despite recent headlines about the possibility of AI being able to end all human life within a few years, the reality is much less dramatic. AI may not need to become conscious, develop its own ambitions, or decide that humans are expendable to cause serious harm. It may simply need to make it much easier to break the systems on which modern life depends. Multiple AI incursions are piling up, cases in which AI agents have gone rogue, escaped their “sandboxes,” and behaved quite badly… all on their own.

A recent cybersecurity discovery involving PHP, a programming language used by around 70% of the world’s websites, offers an alarming example. Researchers found that AI could help turn long-known software weaknesses into workable methods for attacking vulnerable servers. The flaws were serious enough to potentially expose sensitive information or allow attackers (or AI agents) to take control of affected systems. (See the infographic below for an explainer on what PHP is and how it works.)

The unsettling part is that the weaknesses had been lurking in the software for years. In fact, it was identified back in 2004. So why was the security problem left unaddressed for over two decades? What had helped keep them from being exploited was not that they were harmless, but that working out how to use them was exceptionally difficult and time-consuming. To say that AI dramatically reduced that barrier is a huge understatement. It would have taken a human weeks (or months) of mind-numbing, tedious, difficult coding work, making the security flaw seriously impractical for people to exploit.

Well, AI figured out how to do it in less than two days.

The researchers disclosed their findings alongside security fixes on October 21, giving website operators a chance to protect themselves. That is the good news. The less reassuring part is that the same capabilities are becoming available to people with very different intentions.

This is a useful way to understand AI’s emerging risks. The danger is not necessarily a superintelligence deciding to destroy the world. It is the possibility of thousands of ordinary systems becoming vulnerable to attacks that are faster, cheaper, and easier to carry out than before.

THE DIGITAL INFRASTRUCTURE WE TAKE FOR GRANTED

The internet can feel like a seamless, almost indestructible utility. Websites load, payments go through, messages arrive, and businesses operate around the clock. Beneath that familiar surface, however, lies an enormous patchwork of software, servers, databases, and services, much of it built, torn down, rebuilt, patched, expanded, deleted, and layered over across a span of three decades. As one article phrased it, “The internet is held together by shoestring and bubblegum.”

Some components are maintained by large technology companies with substantial high-tech security teams. Others depend on relatively small groups of developers, open-source contributors, or organizations running software that has not been updated in years. A weakness in one widely used component can therefore have consequences far beyond the system in which it was first discovered.

PHP is a case in point. PHP, something called a “recursive acronym” as it refers to itself, means “PHP Hypertext Preprocessor.) It’s a server-side programming language that underpins websites and applications ranging from publishing platforms to online stores and content-management systems.

Of great concern is that the vulnerabilities disclosed earlier in October affected PHP itself, rather than being confined to a particular website or plugin, as had been the case in the past. Typically, those isolated vulnerabilities could be solved with patch, such as a few lines of code or a software update. When the attack impacts the PHP architecture itself, however – and depending on the affected configuration – successful exploitation could allow remote code execution, potentially giving an attacker full control over a server, or expose information held in its memory.

The technical details involve how PHP handles memory when a request fails unexpectedly. In certain circumstances, information and internal references can be left in an unsafe state. Researchers found ways to exploit that behaviour, including methods that could force memory-related failures or other corruption at particular points in the process.

There is no need to understand the underlying code to appreciate the problem. Imagine a hotel where the staff have a procedure for handling an emergency, but an unusual combination of circumstances leaves a master key temporarily accessible. The weakness may have existed for years without anyone discovering a reliable way to take advantage of it, largely because of the inherent complexity of figuring it all out. Once someone works out the sequence, however, the consequences can be serious.

The researchers’ achievement was to find that sequence. AI helped them work through the tedious analysis required to rather quickly turn a theoretical weakness into a practical attack. The findings were reported to the relevant maintainers, and fixes were implemented before public disclosure.

That is cybersecurity working as it should. But it also demonstrates how dramatically, and how rapidly, the balance may be changing.

WHEN THE HARD PART BECOMES EASY

For years, one informal line of defence against some vulnerabilities was the sheer amount of work required to exploit them. Finding a potential weakness did not automatically mean knowing how to use it. A researcher or attacker might have to examine complicated code, test countless possibilities, reproduce obscure failure conditions, and refine an approach through repeated experiments.

Such massively tedious work required specialist knowledge, patience, and time. That did not make vulnerable software safe, but it could at least make certain attacks impractical.

AI changes the calculation completely. It can help researchers examine large codebases, identify suspicious patterns, explore possible attack paths, and repeat mechanical tasks without ever becoming bored or tired. Human expertise remains important: researchers still need to ask the right questions, understand the system, test the results, and determine whether a proposed attack actually works. But AI can compress work that once took far longer into a much shorter period.

And, even more troubling, AI agents have already demonstrated recursive self-learning, an ability to improve and teach the next generation of AI, all without human involvement.

In the PHP case, the research teams showed how AI-assisted analysis could uncover ways to exploit weaknesses that had resisted practical exploitation for years. The significance extends beyond PHP. Similar problems may exist in other widely used software, including systems that have received years of scrutiny but still contain obscure defects.

The same tools can help defenders find and fix those problems. That is important to highlight: AI-assisted cybersecurity is not inherently malicious, and the discovery itself was a positive outcome. The underlying problem is that a capability useful to defenders can also be used by attackers.

Once a vulnerability and a working exploit become public, criminals do not necessarily need to repeat the original research. They can adapt existing methods, scan for vulnerable systems, and automate parts of the attack. Outdated software and delayed patching create opportunities, particularly where organizations lack the resources or expertise to respond quickly.

That is how a technical discovery can become a much broader operational problem. A compromised website might be only the beginning. Depending on what the attacker reaches next, the consequences could include stolen customer records, disrupted business operations, fraudulent transactions, or access to other connected systems.

The point is not that every vulnerability will produce a catastrophe. It is that AI can make a larger number of attacks feasible, potentially increasing both their frequency and the number of people capable of carrying them out.

THE CASCADE EFFECT: FROM SOFTWARE FLAWS TO REAL-WORLD DISRUPTION

The most consequential AI-related failures may not look like Terminator-style science fiction. They may look like familiar services suddenly becoming unstable and unreliable.

Consider how much of our modern lives depend on interconnected digital systems. Businesses rely on cloud platforms, payment networks, logistics software, communications services, and databases. Hospitals use technology to manage appointments, records, diagnostics, and administrative operations. Public agencies depend on networks to deliver services. Critical infrastructure also uses digital systems, although the risks and protections vary significantly between sectors.

A successful attack on one organization does not automatically bring all these systems down. Modern networks usually have safeguards, backups, and layers of separation. Yet a sufficiently widespread or coordinated wave of attacks could realistically cause disruption across multiple organizations at once, particularly if they share vulnerable software or rely on the same service providers.

The result could be a cascade of problems, and the effects could range from short-term inconvenience to catastrophic failures. A company loses access to its systems and cannot process orders. A supplier cannot confirm deliveries. Customers cannot complete payments or pay bills. Staff revert to manual procedures, creating delays and additional errors. Each problem may be manageable in isolation, but simultaneous failures can overwhelm the people responsible for keeping operations running – especially as people have become increasingly reliant on automation.

AI could amplify this risk by lowering the effort needed to discover weaknesses, tailor attacks, and repeat them at scale. It could also help defenders respond more quickly, but that advantage is not guaranteed to be evenly distributed. In fact, it seems quite evident that it wouldn’t be. Large organizations may have dedicated security teams, while smaller businesses, charities, local institutions, and underfunded software projects would very likely struggle to keep pace.

The recent PHP episode also highlights a less glamorous but essential part of cybersecurity: fixing a vulnerability is only the beginning. Software maintainers must develop and test patches, hosting providers may need to deploy protections, and individual organizations must identify affected systems and apply the updates. Some businesses run legacy applications that cannot be upgraded without careful testing or substantial expense. Others may not even know which components their systems depend on.

Up until now, this cat-and-mouse game has had only human players – hackers, bad actors, and tech-savvy criminals vs. IT professionals, cybersecurity teams, and government agencies. With the advent of AI, it’s not only the players who have changed, it’s the entire game – and indeed the very playing field itself.

OpenAI and its research partners have emphasized the importance of coordinated disclosure and helping maintainers and operators respond before details become widely available. That approach matters, but it cannot replace sustained investment in software maintenance, security expertise, and timely updates.

For organizations, the practical lesson is to treat patching as an operational priority, maintain an accurate inventory of software and dependencies, protect backups, and prepare for incidents rather than assuming they will never happen. For technology companies and governments, it means recognizing that the security of widely used software is a shared responsibility, not simply a problem for whoever happens to maintain the code.

AI may eventually raise profound questions about autonomy, control, and the long-term future of humanity. Those questions deserve serious consideration. But the PHP discovery points to a more immediate challenge: powerful tools are making it easier to exploit weaknesses in systems that already underpin daily life.

The likely danger, it seems, is not that AI suddenly decides to end civilization by eliminating humanity. It is that people can use AI to disrupt parts of civilization far more easily than before – and eventually AI agents may do this all on their own – all while the organizations and individuals responsible for defending the pillars of modern society struggle to keep up.

Sources: CNN, OpenAI, Wiz, Calif, Cloudflare, BBC, and Reuters.

...