Dario urges pacing the frontier; ex-OpenAI safety lead says p(doom) is endogenous
soumitrashukla9 · x · 2026-09-14
Anthropic CEO Dario Amodei published a new essay, "We Must Pace the Frontier," arguing the AI industry should slow down, with a three-part plan. Anthropic unilaterally commits to the first step: granting third-party evaluators permanent, employee-level access to its systems to verify safety measures, report incidents, and assess alignment during training.
Quoting the essay, ex-OpenAI safety lead Houda responds that she is less worried about existential risk today than in 2022 despite far more capable models — the biggest uncertainty then, whether labs would seriously invest in alignment, has meaningfully resolved. She argues p(doom) is endogenous, not an exogenous constant: catastrophe probability depends on what labs, governments, researchers and society actually do, so numbers without those assumptions tell us little. She still believes AI's expected benefits outweigh costs, conditional on getting it right.
Related event: Anthropic CEO Dario Amodei Calls for Slowing Down Frontier AI(194 posts)→
More from AGI Musings
- Security researcher calls out AI firms: fix misaligned products, take accountability — samiramanabi · 2026-09-14
- Anthropic's slowdown logic questioned: if China distilled its models, slowing down is pointless — rao2z · 2026-09-14
- The Very Real Threat of a Persistent AI Agent Botnet — ComeToMyFuckingDeli · 2026-09-14
- Virologist slams 'X% chance AI kills everyone' claims as intellectual dishonesty — DeryaTR_ · 2026-09-14
- Domain experts: in fields they know well, LLMs are useful but overhyped — mariofilhoml · 2026-09-14
- banteg's satire: 'microdoom' — trading extinction risk on Uniswap — banteg · 2026-09-14