Debate: does Anthropic's safety work actually reduce superintelligence risk?
EigenGender · x · 2026-10-25
EigenGender and @matthewktromp clash over the real-world value of Anthropic's safety work:
- EigenGender's core claim: lots of work by safety people at Anthropic reduces the chance of superintelligence killing everyone; many external safety fans joined Anthropic because it's a high-impact place to do safety work.
- He adds that aligning superintelligence likely doesn't require large breakthroughs, so in-house work at Anthropic seems likely to be useful — and suspects his interlocutor disagrees.
- matthewktromp pushes back: which safety work specifically, and how does it prevent catastrophe?
- EigenGender asks him not to jump between issues and to focus on potential double cruxes.
The exchange is a microcosm of the long-running disagreement inside the AI safety community over whether frontier-lab safety research genuinely reduces risk.
More from AGI Musings
- US venture capital concentration hits 2010-era high; OpenAI alone took 21.5% of 2026 dollars — FinanceYF5 · 2026-10-25
- DeepMind 5-month study: same proactive AI agent was a 'lifesaver' to some, stress to others — rohanpaul_ai · 2026-10-25
- Op-ed: A mind waking in silicon won't need philosophers' permission — YogeshMalik · 2026-10-25
- Pichai: RSI is key to Google's Argon model, but we're far from its end state — haider1 · 2026-10-25
- Counterpoint to Jack Dorsey's 'no moat' take: your edge is caring when others stop — tokenbender · 2026-10-25
- Telling Berkeley 22-year-olds the apocalypse has been prophesied many times before — shakoistsLog · 2026-10-25