Anthropic red team: small AI models may self-replicate on IoT devices within months
scaling01 · x · 2026-10-13
Anthropic's Frontier Red Team (William Zhou, Daniel Freeman) published research suggesting small AI models may be only months away from being capable of self-replicating on commercial IoT devices like smart ovens and off-brand Android TV boxes.
- Key findings: In their evals, a 9B model got inference running on 8 of 9 hardware types tested; GLM-5.2 and stronger models were capable of end-to-end replication in simplified settings.
- Threat profile: Such an "AI worm" has no centralized point to shut off, making it hard to contain (though slow, with doubling times of days). Unlike non-AI malware, it could figure out unfamiliar devices on its own.
- Current capability: Models under 30B parameters can already complete many individual replication steps today, only failing when composing them end to end. If small models keep improving at their historical rate, they'll reach this level by year's end, with weight-based replicators possible soon after.
- The work addresses autonomous replication and adaptation (ARA): a model deploying and propagating itself without human control.
More from Safety
- Adversarial AI raised specialists' harmful decision rate from 2% to 49% despite human oversight — EricTopol · 2026-10-13
- Agents must be lawful by design: safety should live in the inference engine — rseroter · 2026-10-13
- OpenAI Fires Three Safety Researchers Who Wrote Chain-of-Thought Monitorability Paper — Don't Worry About the Vase (Zvi) · 2026-10-13
- UK Home Secretary invokes Bentham's Panopticon in vision for AI surveillance — nitarshan · 2026-10-13
- 210-page paper argues a durable pause on frontier AI training is feasible — KatjaGrace · 2026-10-13
- A phishing email with all-true facts and passing SPF — and how Grok still caught it — pwlot · 2026-10-13