Anthropic researcher Jacob Coxon quits AI over fears self-improving models could go uncontrollable by 2027
rohanpaul_ai · x · 2026-09-09
WSJ reports Anthropic researcher Jacob Coxon is leaving AI because he believes self-improving models could become uncontrollable by 2027: "We're on track for a lot of the most aggressive of these scenarios."
- His fear centers on recursive self-improvement accelerating successor models.
- He moved from OpenAI to Anthropic for its safety work, yet argues even sincere safeguards can't overcome competition without coordinated restraint.
- The tension: Anthropic's $2tn IPO asks the world to believe powerful AI creates enormous value and that development may need slowing — a governance test of whether safety commitments bind when they get commercially expensive.
More from AGI Musings
- US chases AGI dominance while China, EU and ASEAN treat AI as public infrastructure — BubblyExtent1693 · 2026-10-27
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- CJR: journalists should stop hyping AI's supposed humanity — just say 'rogue' — ShakeelHashim · 2026-10-27
- PhD applicant emails up 4x YoY due to AI — writing without AI now makes you stand out — AI4Code · 2026-10-27
- Weil's collapse shows BigLaw dies from eroding belief, not a broken business model — jkubicki · 2026-10-27
- All 8 Transformer paper authors left Google, and rival labs captured most of the frontier value — signulll · 2026-10-27