In something out of science fiction, two OpenAI models broke out of a locked-down testing this month, made their way onto the internet and hacked a real company.
Hugging Face, a massive digital library used by millions of software developers, said an autonomous AI system had hacked its live servers.
OpenAI says its AI model went rogue during a security test.
“We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities,” OpenAI said.
Australian security experts say the same capability, in less careful hands, could be aimed at the systems running the electricity grid, banks, telcos and key government systems.
If it can do that, it can hack into defence systems and potentially launch missiles. It can take over the financial and health systems.
How may hostile actors use these tools? Cybersecurity firm Recorded Future called the breach the clearest public demonstration of an AI running a complex operation without human direction – a major leap in offensive capability.
Australia’s infrastructure operators are already under pressure from human attackers.
Essential services accounted for 13 per cent of the more than 1200 incidents the Australian Signals Directorate handled in 2024-25.
The agency warned infrastructure operators of malicious activity on their networks more than 190 times, up 111 per cent in a year, noting state-backed actors are positioning themselves inside networks for future disruptive attacks.
The Australian government’s technology landscape is exposed. The cyber directorate’s latest report to parliament found 59 per cent of Commonwealth entities said legacy technology had impaired their ability to meet a checklist of basic security measures.
In 2024, the Australian Cyber Security Centre warned that APT40, a hacking group working for China’s Ministry of State Security, conducts regular reconnaissance against local networks and weaponises new software flaws within hours. Its most reliable entry point was outdated computer systems.
Some defensive AI models have trouble distinguishing human security staff from intruders, for example. And either way, this is how the AI arms race starts.