Ex-OpenAI/Anthropic Researcher Jacob Coxon Resigns Publicly, Warning of Superintelligence Risks

On Sept 9, Jacob Coxon, a 27-year-old researcher who spent three years on pretraining at OpenAI and then Anthropic, announced his resignation from Anthropic on X, accusing both labs of "racing straight toward self-improving superintelligence, gambling with our lives" and of acting irresponsibly. His post drew roughly 170 million views; follow-up interviews with ABC and BBC amplified his warnings, and Anthropic alignment lead Evan Hubinger's agreement that the probability of AI-driven human extinction within a decade exceeds 10% fueled debate inside major AI companies and across the public sphere.

Confirmed

• Coxon worked on pretraining at both OpenAI and Anthropic and resigned publicly on Sept 9 citing AI risk; he was a hands-on researcher, not a marketing or executive role.
• In an ABC interview he likened building AI to "summoning an alien mind we don't fully understand," saying the idea alone is frightening even without a concrete scenario.
• He argued AI may be the most dangerous technology humans have created—a superweapon that cannot be controlled like nuclear weapons—and that the current race is leading humanity toward extinction.
• In a BBC interview, Coxon said that without slowing down, "it is very likely we will all die in the near future" is not an exaggeration; many insiders who write frontier models carry a judgment of over 10% probability of human extinction daily, and are "genuinely terrified for humanity's future."
• Evan Hubinger, Anthropic's head of alignment science, agreed with the >10% extinction-within-a-decade estimate; critics such as firstadopter attacked the framing as incompatible with "controllable risk," while Andy Hall's Substack essay "The Makers of our Fate" explained why he rejects the imminent-extinction view.
• NYT reporter Mike Isaac reported snowballing superintelligence-risk discussions inside AI companies, saying the event triggered fierce internal debates about doom risk at four major firms.
• A physics-background writer (relayed by @AryHHAry) argued AI's danger is not novel in kind but jumps several orders of magnitude in scale: past dangerous technologies had physical bottlenecks as "speed bumps" (costly visible uranium enrichment, BSL labs, chemical supply chains), whereas information has no mass and faces almost no physical constraint on copying and spread.

Unconfirmed

• The claim that Anthropic leadership cited a >10% extinction probability in a response mainly comes from a relay post (m10); the original source and exact wording, including the precise context of Hubinger's statement, remain unverified.

Why it matters

• As an insider who trained models himself, Coxon's public departure pushed the debate over whether labs truly prioritize safety into public view, the latest case of an insider voice on loss-of-control risk.
• The episode galvanized discussion of the self-improving superintelligence race and existential risk, with internal debates at major AI companies; Hubinger's stance shows such concerns are not limited to departed staff, while also drawing criticism that a "10% extinction probability" framing admits the risk is not controllable.
• The physical-bottleneck argument offers an accessible case for why AI risk differs from nuclear and biological weapons, supplementing this debate.

2026-09-13 ~ 2026-09-15 · 14 related posts

Full story(6 episodes)→

• Episode 1: Ex-Anthropic/OpenAI researcher warns AI may escape human control(2026-09-12, 2 posts)
• Episode 2: Ex-OpenAI/Anthropic Researcher Jacob Coxon Resigns Publicly, Warning of Superintelligence Risks(2026-09-13, 14 posts)
• Episode 3: Guardian Podcasts Examine Whether AI Could Destroy Humanity(2026-09-14, 2 posts)
• Episode 4: Ex-Anthropic researcher slams leadership's racing logic in AMA(2026-09-16, 3 posts)
• Episode 5: Anthropic Researcher Jacob Coxon Quits, Warning AI Industry Is Gambling with Lives(2026-09-17, 2 posts)
• Episode 6: Anthropic engineer resigns, warning superintelligence race gambles humanity(2026-09-19, 3 posts)

• [source] Ex-Anthropic researcher's exit post hits 170M views as Hubinger admits >10% doom odds — adamamcbride · 2026-09-13
• Anthropic model trainer Jacob Coxon quits, calling AI an uncontrollable super weapon — victor_explore · 2026-09-13
• Andy Hall: Why He Doesn't Buy Imminent Extinction, as Anthropic Exits and 10%+ Warnings Mount — soumitrashukla9 · 2026-09-13
• [source] Ex-Anthropic Researcher Jacob Coxon Tells BBC Insiders Are 'Genuinely Frightened' — rohanpaul_ai · 2026-09-14
• Ex-Anthropic Researcher on BBC: Insiders See Over 10% Extinction Risk Daily — rohanpaul_ai · 2026-09-14
• Anthropic Researcher Quits Over AI Risks as Doomsday Debate Snowballs Across OpenAI, Meta, Google — RebeccaBellan · 2026-09-14
• Ex-OpenAI/Anthropic employee Jacob Coxon quits, likens building AI to 'summoning an alien species' — AlexTensor · 2026-09-14
• Researcher resigns from Anthropic, slamming OpenAI and Anthropic's superintelligence race — vkrakovna · 2026-09-14
• Anthropic alignment lead agrees AI "kill all humans" odds exceed 10% in next decade — firstadopter · 2026-09-14
• Anthropic researcher resigns warning self-improving AI could threaten humanity — emmanuelvivier · 2026-09-14
• Physicist's case: AI risk jumps orders of magnitude because information has no mass — AryHHAry · 2026-09-15

3 near-duplicate retellings: kevinsurace · AryHHAry · Intelligent-Lynx-953