Alignment researcher fears current methods won't scale to superintelligence
hughbzhang · x · 2026-09-10
Alignment researcher Hugh Zhang says his biggest current fear is that the methods used to align models today will not scale as models approach superintelligence, while acknowledging deep disagreement and saying he'd be happy to be proven wrong. His interlocutor pushes back: no one expects a formal system of 'perfect' alignment, and in empirical terms we already know a lot about aligning today's systems.
More from AGI Musings
- 'Half the sites I try via agents block access' — the anti-bot paradox holding back the agentic economy — XFreeze · 2026-10-27
- Cosmos Institute launches Oxford AI philosophy journal with $10,000 best-paper prize — lawhsw · 2026-10-27
- Stop remaking old games with AI: a call for more ambitious agentic software — rschu · 2026-10-27
- US chases AGI dominance while China, EU and ASEAN treat AI as public infrastructure — BubblyExtent1693 · 2026-10-27
- Lab monkeys at $30-80K each: a 4-year breeding cycle is the real preclinical drug bottleneck — zebird0 · 2026-10-27
- CJR: journalists should stop hyping AI's supposed humanity — just say 'rogue' — ShakeelHashim · 2026-10-27