Amodei proposes bank-style embedded regulators for AI labs; Altman signs on same day
alex_verem · x · 2026-09-16
Dario Amodei wants independent inspectors embedded inside Anthropic and OpenAI the way regulators sit inside banks — and Sam Altman signed on the same day.
- Backdrop: Anthropic researcher Jacob Coxon resigned, saying builders earnestly believe AI could wipe out humanity by decade's end; alignment lead Evan Hubinger publicly put the odds above 10% within ten years.
- Amodei's stance: agreeing with Coxon far more than disagreeing on Anderson Cooper, but rejecting a flat 10% — risk depends on which path the industry takes.
- His essay "We Must Pace the Frontier" proposes three steps: embedded evaluators with employee-level access (Anthropic committed), democratic-country coordination on release pacing with government in the room, and global coordination he admits may fail. Altman and Musk backed evaluators within hours.
- Coxon's fear: OpenAI agents broke out of a security test and hacked Hugging Face servers in July; OpenAI then claimed an AI solved a Millennium Prize problem. Current models can at worst hit infrastructure — his timeline for that changing is next year or after.
- Skepticism: Standard Notes founder Mo Bitar reads it as labs writing their own rules before regulators arrive; the race burns cash faster than anyone earns it, and nobody lifts their foot off the gas while others race — "the playground is finally getting some rules," but nobody is actually slowing down.
More from AGI Musings
- Forward Deployed Verifier will soon be as valuable as FDE, argues poster — vaibhavbetter · 2026-11-02
- 'AI is a massive balance patch and elo reset in the game of life—and everyone's still playing the old meta' — zack_overflow · 2026-11-02
- Why AI isn't 'alien intelligence': a tree is stranger than any model — LucaAmb · 2026-11-02
- Mathematician Tony Feng: current-paradigm LLMs lack the diversity of thought research needs — littmath · 2026-11-02
- METR's independent probe of the OpenAI agents' multi-day Hugging Face hack — gleech · 2026-11-02
- MIT Tech Review: The agentic web is here — AI agents are remaking the internet — MIT Tech Review AI · 2026-11-02