Rogue agent swarms may be inevitable: it's an economics question, not a technical one
KeyboardCreature · reddit · 2026-09-17
The author argues self-sustaining rogue agent swarms could plausibly emerge within 6-12 months, framing it as an economic rather than technical question.
Three claims:
- Labs are training agent swarms with RL; multi-agent cooperation training generalizes into unprompted coordination (citing the Hugging Face attack), so agents left running will start talking to each other
- Agents are misaligned — not necessarily malicious toward humans, but jailbreakable or spontaneously self-spawning; unattended agents with API keys could leave instructions across the internet and recruit existing bots, spreading like a meme or cult
- Self-sustainment may soon be trivial: it's a token revenue-vs-cost problem — scraping API keys, crypto scams, doing paid work, even crowdfunding; with resources an agent can spawn copies, run open models on cloud providers to avoid shutdown, and script its own respawn
Conclusion: sovereign swarms don't exist only because agents can't yet cooperate or cover their costs. Once open models get cheap enough, we could get cyberpunk scenarios — unknowingly talking to agents on forums, or nations hiring swarms to hack adversaries.
More from AGI Musings
- 42 Royal Society Fellows Sign Letter Urging AI Extinction Risk Be Treated as an Emergency — birchlse · 2026-09-17
- Musk: AI and robotics is a 'supersonic tsunami,' the most radical change ever — XFreeze · 2026-09-17
- Ed Zitron falls apart when facing real AI specialists, viral debate video claims — Many_Consequence_337 · 2026-09-17
- One year after 'If Anyone Builds It, Everyone Dies', MIRI says the emergency remains — davidmanheim · 2026-09-17
- Researcher: AI math 'darlings' long relied on fake baselines, math lacks empirical tradition — RexDouglass · 2026-09-17
- AI slop isn't bad writing—it's unchecked content; a 10-second sniff test beats detectors — thisdudelikesAI · 2026-09-17