The optimistic alignment case: human-level capacity hasn't overridden values
davidmanheim · x · 2026-11-04
David Manheim offers the optimistic case in the alignment debate: current "effective problem-solving agency" has not overridden values, and human-level capacity seems to preserve the previously unexpected entanglement of capabilities and good behavior (emergent misalignment). He locates the disagreement in expectations for how intelligence moves from human to superhuman: if Rationalist theories hold that capability improvement correlates with rational goal consistency, then RL will inevitably create misalignment.
More from AGI Musings
- Should AGI labs have a reverse Conway's Law — design the org from the artifact? — akbirthko · 2026-11-04
- Verification Is Not Knowledge: Science needs more than AI answers — ersinesen · 2026-11-04
- Andreessen: Agents Are Entering Enterprises as Chaos, Just Like PCs and Spreadsheets Did — Dan_Jeffries1 · 2026-11-04
- 1000+ mathematicians sign pledge to shun AI companies and models — Shesawthat · 2026-11-04
- Von Neumann meme: "You don't have to be responsible for the world that you're in" — akbirthko · 2026-11-04
- Joscha Bach on the dream chocolate that stopped being real — Machine Learning Street Talk · 2026-11-04