Researchers push back on 'agent swarm' takeover premise: real-world 80% AI success is only ~15-minute tasks
rbhar90 · x · 2026-11-18
Dwarkesh Patel shared Paul Christiano's AI-takeover scenario—a rogue internal agent swarm seizing control during the intelligence explosion, exfiltrating billions of superintelligent copies that compromise global infrastructure. Nate Witkin rejects the first premise: predicting 'agent swarms better than any human intelligence agency' is out of step with evidence from top benchmarks, agent-coordination research, and economic proxies. In practice, agents one-shot very few task types, mostly those with loose success conditions. Attached OpenAI data (modified by Ramez Naam) shows real-world 80% task success without human intervention sits at only 15 minutes; for 64-128h tasks, zero-intervention success is just 16% with 36% failures/tool errors. Capability gains accelerate over time (15.4→29.3 points/year) but diminish per R&D dollar.
More from AGI Musings
- Peking U Paper: RSI Tax Shows Verification Eats 74.8-93.6% of Self-Improvement Cost — RobertTLange · 2026-11-18
- SF AI circle's undercurrent of anxiety: only 1-5 years of intellectual alpha left before AGI — CatAstro_Piyush · 2026-11-18
- The nonhuman economy: AI compute buildout may crowd out land, labor and everything else — thomasahle · 2026-11-18
- AI weather model gives hurricane targets one extra day of warning, Nature paper shows — davidmanheim · 2026-11-18
- Software singularity must be a sigmoid: compute substrate caps AI takeoff curves — tszzl · 2026-11-18
- Svwang1: AI bots' fake prosperity is just the credit bubble with a new prop — Svwang1 · 2026-11-18