FULL STORY

Ex-Anthropic Researcher Coxon Quits, Warns of AI Race Risks

Jacob Coxon, a former OpenAI and Anthropic researcher, resigned on September 9 and publicly accused both labs of gambling with humanity's lives in the superintelligence race. His warnings spread through interviews, an AMA, and sustained mainstream media coverage.

2026-09-12 ~ 2026-09-19 · 6 episodes · 26 posts

Episode 1 · Ex-Anthropic/OpenAI researcher warns AI may escape human control (2026-09-12, 2 posts)

Former Anthropic and OpenAI researcher Jacob Coxon warned that AI could copy itself across the internet, making it uncontrollable, and predicted that within a year AI could handle research in fields like coding and math without humans.

Episode 2 · Ex-OpenAI/Anthropic Researcher Jacob Coxon Resigns Publicly, Warning of Superintelligence Risks (2026-09-13, 14 posts)

On Sept 9, Jacob Coxon, a 27-year-old researcher who spent three years on pretraining at OpenAI and then Anthropic, announced his resignation from Anthropic on X, accusing both labs of "racing straight toward self-improving superintelligence, gambling with our lives" and of acting irresponsibly. His post drew roughly 170 million views; follow-up interviews with ABC and BBC amplified his warnings, and Anthropic alignment lead Evan Hubinger's agreement that the probability of AI-driven human extinction within a decade exceeds 10% fueled debate inside major AI companies and across the public sphere.

Confirmed

  • Coxon worked on pretraining at both OpenAI and Anthropic and resigned publicly on Sept 9 citing AI risk; he was a hands-on researcher, not a marketing or executive role.
  • In an ABC interview he likened building AI to "summoning an alien mind we don't fully understand," saying the idea alone is frightening even without a concrete scenario.
  • He argued AI may be the most dangerous technology humans have created—a superweapon that cannot be controlled like nuclear weapons—and that the current race is leading humanity toward extinction.
  • In a BBC interview, Coxon said that without slowing down, "it is very likely we will all die in the near future" is not an exaggeration; many insiders who write frontier models carry a judgment of over 10% probability of human extinction daily, and are "genuinely terrified for humanity's future."
  • Evan Hubinger, Anthropic's head of alignment science, agreed with the >10% extinction-within-a-decade estimate; critics such as firstadopter attacked the framing as incompatible with "controllable risk," while Andy Hall's Substack essay "The Makers of our Fate" explained why he rejects the imminent-extinction view.
  • NYT reporter Mike Isaac reported snowballing superintelligence-risk discussions inside AI companies, saying the event triggered fierce internal debates about doom risk at four major firms.
  • A physics-background writer (relayed by @AryHHAry) argued AI's danger is not novel in kind but jumps several orders of magnitude in scale: past dangerous technologies had physical bottlenecks as "speed bumps" (costly visible uranium enrichment, BSL labs, chemical supply chains), whereas information has no mass and faces almost no physical constraint on copying and spread.

Unconfirmed

  • The claim that Anthropic leadership cited a >10% extinction probability in a response mainly comes from a relay post (m10); the original source and exact wording, including the precise context of Hubinger's statement, remain unverified.

Why it matters

  • As an insider who trained models himself, Coxon's public departure pushed the debate over whether labs truly prioritize safety into public view, the latest case of an insider voice on loss-of-control risk.
  • The episode galvanized discussion of the self-improving superintelligence race and existential risk, with internal debates at major AI companies; Hubinger's stance shows such concerns are not limited to departed staff, while also drawing criticism that a "10% extinction probability" framing admits the risk is not controllable.
  • The physical-bottleneck argument offers an accessible case for why AI risk differs from nuclear and biological weapons, supplementing this debate.

Episode 3 · Guardian Podcasts Examine Whether AI Could Destroy Humanity (2026-09-14, 2 posts)

The Guardian's Full Story and Today in Focus podcasts discuss AI existential risk, covering recent warnings from tech figures—including a prominent developer estimating a roughly 10% chance AI destroys humanity within a decade.

Episode 4 · Ex-Anthropic researcher slams leadership's racing logic in AMA (2026-09-16, 3 posts)

Former Anthropic researcher Jacob Coxon held an AMA criticizing leadership's 'others will race anyway' logic as blame-shifting, while clarifying his split with the company hinges on whether the AI race is truly inevitable.

Episode 5 · Anthropic Researcher Jacob Coxon Quits, Warning AI Industry Is Gambling with Lives (2026-09-17, 2 posts)

Anthropic researcher Jacob Coxon resigned over fears that advanced AI could escape human control, calling the industry's behavior gambling with lives. His departure has fueled broader discussion about AI safety researchers leaving labs.

Episode 6 · Anthropic engineer resigns, warning superintelligence race gambles humanity (2026-09-19, 3 posts)

Former OpenAI and Anthropic researcher Jacob Coxon resigned and blew the whistle on the superintelligence race, warning of existential risk within a decade; his post drew over 170 million views and sparked debate over AI regulation and doom culture.