Ex-OpenAI/Anthropic pretraining researcher resigns, warns labs are gambling on self-improving superintelligence
ayushtweetshere · x · 2026-09-10
Jacob Coxon, who spent three years doing pretraining research across OpenAI and Anthropic, has resigned from Anthropic with a blunt warning: both labs are "racing straight to self-improving superintelligence and gambling with our lives."
Key points:
- He claims people building these systems privately believe AI "could kill us all by the end of the decade"
- Development continues anyway because each lab believes it must reach superintelligence first and can't trust competitors to act responsibly
- The poster frames this as a coordination failure: every participant can see the danger, hire sincere safety researchers, and still rationally choose to accelerate
The thread also notes how "if we don't build it, someone else will" has historically justified terrible decisions.
More from AGI Musings
- Alpha School opinions are 'something of an IQ test,' argues Eric Jorgenson amid AI education debate — RachelVT42 · 2026-09-10
- Anthropic Report Says Coders May Need to Learn to Plumb Toilets — Polymarket · 2026-09-10
- ARK analyst lumps AI doomerism with NIMBYism and anti-GMO: 'elitism disguised as precaution' — skorusARK · 2026-09-10
- Geoffrey Hinton tells CNN AI companies have 'no idea' how to control the technology — pstAsiatech · 2026-09-10
- Runway CEO: Everyone's first instinct with a new frontier model is simulating reality — c_valenzuelab · 2026-09-10
- A first-year PhD student says ML systems research has been made obsolete by LLMs — kohjingyu · 2026-09-10