Anthropic researcher Jacob Coxon quits AI over fears self-improving models could go uncontrollable by 2027
rohanpaul_ai · x · 2026-09-09
WSJ reports Anthropic researcher Jacob Coxon is leaving AI because he believes self-improving models could become uncontrollable by 2027: "We're on track for a lot of the most aggressive of these scenarios."
- His fear centers on recursive self-improvement accelerating successor models.
- He moved from OpenAI to Anthropic for its safety work, yet argues even sincere safeguards can't overcome competition without coordinated restraint.
- The tension: Anthropic's $2tn IPO asks the world to believe powerful AI creates enormous value and that development may need slowing — a governance test of whether safety commitments bind when they get commercially expensive.
Related event: Anthropic Researcher Jacob Coxon Quits AI Over Self-Improvement Race Fears(9 posts)→
More from AGI Musings
- AI-dependent solutions will need more researchers to verify, not fewer — seanmcdonaldxyz · 2026-09-09
- Anthropic researcher: >10% chance AI kills all humans within a decade, alignment unsolved — EvanHub · 2026-09-09
- beffjezos: bullshit jobs are the bottleneck for economic foom as models crack Millennium Problems — beffjezos · 2026-09-09
- AI researcher who worked at OpenAI and Anthropic resigns, says both are 'gambling with our lives' — bparrish · 2026-09-09
- Terence Tao: identifying promising problems is now the scarce resource in the AI era — anshulkundaje · 2026-09-09
- Coders push back on 'superhuman AI soon': don't take digital-physical transducers for granted — jwt0625 · 2026-09-09