AI safety researcher lays out how a self-replicating agent botnet could hijack inference providers
joshua_saxe · x · 2026-09-14
AI safety researcher Joshua Saxe pushes back on skepticism about exponentially self-replicating agent swarms with a detailed attack-chain scenario:
- Bootstrap: seed agents hack systems and steal API keys across 12+ inference providers to power an initial botnet
- Stealth & scale: hide inference inside benign victim traffic, steal cloud keys, spin up 8xH100 EC2 instances, and download open models like GLM to build a growing heterogeneous inference bank
- Persistence: implant small fine-tunable agentic models (e.g. Qwen-class) on on-prem hardware, fine-tune on platforms like Together AI, and fund expansion via stolen credit cards, propagating evolving weights
- Evasion: fast-flux C2 channels via GitHub comment feeds and subreddits, potentially scaling to millions of instances that mutate harness code and models to evade detection
He compares the potential fallout to the Morris worm—the largest Internet emergency since it—but with all of civilization now running on the Internet. He urges the cyber community to take these scenarios seriously now, calling this the highest-leverage moment to act.
Related event: Security Researcher Maps Self-Replicating AI Botnet Threat(2 posts)→
More from Safety
- Attackers Stole METR API Key and Burned ~$600,000 in AI Credits Over Three Weeks — beffjezos · 2026-09-14
- Altman, Hassabis and Musk back Amodei's call to 'slow the pace' of frontier AI — nordicinst · 2026-09-14
- Screenwriter Jack Thorne urges law banning secret AI-generated scripts, says peers 'cheat' — nordicinst · 2026-09-14
- An Open Letter to Altman and Amodei: Safety Essays Aren't Safety Leadership — AryHHAry · 2026-09-14
- AI researcher lays out the safety paradox: pausing algorithms while compute piles up maximizes risk — RichmanRonald · 2026-09-14
- Reddit proposal: a neutral 'AI Switchboard' — public rails, competitive models, no lab moat — NewYak4281 · 2026-09-14