Alignment researcher: the word "aligned" implies consistencies that don't exist
davidmanheim · x · 2026-11-02
AI safety researcher David Manheim critiques the term "alignment" in a debate with ASI advocates: the problem isn't "whose values," but that the concept implies consistencies that simply don't exist — extrapolating poorly understood intuitions to new scenarios and calling the result "aligned." The quoted interlocutor had argued any plausible alignment consensus beats the chaotic whims of flawed human leaders, to which Manheim replies that "aligned" is being used as if it means something.
More from AGI Musings
- 'Vibe distribution' is the problem to solve now that anyone can prompt an agent — pjausovec · 2026-11-02
- Yoshua Bengio warns frontier AI labs lack convincing safety plans at Berlin summit — Yoshua_Bengio · 2026-11-02
- Researcher resigns from JMLR, arguing AI reviews now beat rushed human peer review — josephdviviano · 2026-11-02
- "They Were Incapable of Coordination": Manheim on CEOs' Broken AI Pause Pledges — davidmanheim · 2026-11-02
- Luiza Jarovsky on the AI safety crisis: tech, law, and regulation must advance together — LuizaJarovsky · 2026-11-02
- "Causal patterns are all you need": MLStreet Talk debates whether the hard problem exists — burny_tech · 2026-11-02