roon hails an interpretability breakthrough; Eliezer fires back: aligning ASI is no normal science
jd_pressman · x · 2026-10-16
A clash over an interpretability breakthrough:
- Nick Cammarata says interpretability "might be the most important scientific problem in the world" and hints he has good news
- roon says Nick was underselling it: "this may be remembered as a major moment in scientific history... news so good even eliezer will have to lower his pdoom. We can start making alignment into a 'normal science'."
- Eliezer Yudkowsky pushes back hard: if you think any level of interpretability makes aligning ASI "normal science" rather than "maaaybe possible," you don't understand or believe in ASI
jdpressman notes he's skeptical of the object-level claim itself.
More from AGI Musings
- Testing alongside an agent in a p2p room feels oddly real, says dev — sull · 2026-10-16
- Yudkowsky's 2008 takeoff scenarios resurface as AI-advances-AI takes hold — jd_pressman · 2026-10-16
- Guardrails can make risky delegation work — but it takes a full-time team, says iandanforth — iandanforth · 2026-10-16
- e/acc founder: agentic models are autocomplete for digital actions — beffjezos · 2026-10-16
- What can you truly delegate? Only tasks with unambiguous, verifiable results — iandanforth · 2026-10-16
- Casper: Understanding 90% of neurons ≠ understanding the network — StephenLCasper · 2026-10-16