Anthropic's AI Warning Is a Chilling Case of Déjà Vu
Are vast numbers of smart devices (refrigerators, stoves, vacuum cleaners, lawn mowers, security cameras and televisions) plotting to take over the internet? No. But they could soon host self-replicating AI models that spread from one device to another and are very difficult to contain. Welcome to the disturbing world of AI worms, where, […]
Posts in this thread will be added to your daily email summary and home page feed.
Anthropic says its research has found "signs of life for AI worms" that can colonize home devices. Experts say they have a point.
Posts from this author will be added to your daily email digest and home page feed.
is a London-based reporter at The Verge covering all things AI and a Tarbell Senior Fellow. Previously, he wrote about health, science and technology for Forbes.
Are a multitude of smart devices—refrigerators, ovens, vacuum cleaners, lawnmowers, security cameras, and televisions—conspiring to take over the Internet? no But they could soon host self-replicating AI models that spread from machine to machine and are very difficult to contain.
Welcome to the tumultuous world of AI worms, where, depending on who you ask, the prospect of models spreading out of control in home appliances is either a looming cybersecurity risk or another case of AI companies spreading malicious ads. In a blog post published this week, Anthropic warned that small AI models could be "months" away from achieving such capabilities. Its experiments provide evidence to support this, but leave a large gap between what researchers have shown and the threat they predict. However, experts who spoke to The Verge say that while computer worms and fears of self-replicating AI are nothing new, recent developments show that there are good reasons to take the warnings seriously.
Self-replicating machines have long intrigued AI safety researchers. They raise the prospect of expanding systems beyond human control that are difficult, if not impossible, to shut down. As with many fears about the advancement of AI, attention has largely focused on the more powerful frontier models, such as OpenAI's Astra or Anthropic's Mythos. But a key limiting factor is their size: they're huge. Boundary models that potentially contain trillions of parameters—values that are learned during training—require vast computing resources, largely confining them to data centers, where they can be more easily monitored and, in principle, turned off.
However, smaller models present a different problem. They are rapidly improving, can run on consumer-grade hardware, and are often freely available for download. Entropy's researchers wanted to know if it could run on a network of smaller, less powerful devices. They successfully split Google's Gemma 4, a model with 26 billion parameters, into 15 Android smart TV boxes, none of which are powerful enough to run it. It worked, albeit very slowly.
Inside the suddenly explosive world of artificial intelligence safety
A company is at the center of a wave of rogue AI attacks
Why can't we keep rogue AI off the internet?
Having shown that a model could be run on home devices, the researchers wanted to see if AI could perform the necessary actions for it or replicate itself in such an environment. They built a simulated network of computers designed to behave like Android devices, then released an AI-based "worm" — a type of malware that automatically replicates and spreads across networks — whose job it was to find new machines, copy the AI model to them, and run it on each. It is worth noting that the model being released - the 8 billion meta parameter Llama 3.1-8B - was different from the model coordinating the attack and was much smaller.
Only the most complex model tested, GLM-5.2 from China, was able to propagate through the entire network, although smaller models successfully passed individual steps. Based on the speed of their improvement, Anthropic researchers predict that smaller models will be able to match the capabilities of GLM-5.2 by the end of the year.
"As an extrapolation from our experiments, we expect the future to be flooded with agents operating on all types of hardware, including some that are not under human control," the researchers wrote. They admit that they have yet to demonstrate "a perfect rogue replicator," nor do they know what that might look like (they suggest "a "distributed digital slime mold" or "a swarm of fruit flies"). But they said their findings showed "signs of AI worms alive" and called for stronger security on the many devices a replicating system could use to spread.
"We must try to harden our systems to make it difficult for such replicators to spread. Prepare for a world where they are here to stay." they wrote It's important to note that Entropic did not use its own models for these tests, and has repeatedly argued that open alternatives to its AI models are "higher risk" than closed systems.
The warning has sparked familiar accusations of PR hype and the destruction of AI, including from readers of The Verge's previous coverage. After all, there's a significant gap between showing that an AI can run smoothly on a bunch of cheap TV boxes and a replicating AI worm tearing through home devices. But while questioning a company's motives is certainly worthwhile, claims of intentional fear-mongering are hard to prove, especially since Anthropic has long raised concerns about AI safety. Nor is Entropic alone in exploring the threat of self-replicating AI or calling for better security in connected devices.
Earlier this year, Palisade Research, a California-based AI safety research firm, demonstrated that AI models can autonomously copy themselves onto other machines. "What's new here is that these models were much smaller in number of parameters than previous models that have been shown to be reproducible," Palisade CEO Jeffrey Ladish told The Verge. Entropy's work goes a step further by showing that an AI can run on multiple low-power devices. "This is also true for large scale models," Ladish said, predicting that frontier models may eventually be able to run on large networks of consumer chips, such as those found in MacBooks.
Previous research has shown that the threat of artificial intelligence-based computer worms is real.
Computer worms are not new either. They have been contaminating cars for decades. But AI can make them smarter, more adaptable and better equipped to deal with any obstacle. Ladish noted that even smaller models are getting better at conducting cyberattacks independently. "Previous research has shown that the threat of artificial intelligence-based computer worms is real," University of Toronto professor Nicholas Papernot, who has studied them, told The Verge.
Taken together, the capabilities Anthropic has demonstrated could dramatically expand the range of devices worth targeting in cyberattacks. "Devices that were previously untargeted due to the limited resources and time of human attackers can now become targets, if not for the information they possess, to use as a launching pad to attack more valuable targets," says Papernot. And there's no shortage of targets: the Internet is full of poorly secured smart devices.
Papernot warned against humanizing AI, but said the threat of AI-based computer worms is "concrete" and "deserves immediate action", especially when it comes to securing critical infrastructure such as hospitals and power grids.
Unless there's a massive effort to secure devices of all kinds, it's likely that artificial intelligence worms will do it. We have fertile ground for colonization, Ladish said. I expect to see the first of these in the wild within the next 12 months.
I tried Apple's HomeView smart display
Everything Apple released at its "Welcome Home" event
Her reps say it's "incomprehensible" that Instagram allowed the artificial intelligence image of Mia Goth to go viral.
A free daily digest of the news that matters most.
By providing your information, you agree to our Terms of Use and Privacy Policy. We use vendors who may also process your information to help provide our services. This site is protected by reCAPTCHA and Google's Privacy Policy and Terms of Service apply.