Why is it harder to verify alignment than capability? A thread on evaluation asymmetry
sandersted · x · 2026-11-12
sandersted asks why we have less confidence in alignment generalizing than capabilities generalizing. His follow-up example: judging whether an action is 'successful' is easier than judging whether it's 'aligned' — therapy-style life advice can't be validated by randomized controlled experiments, yet lacks clear red lines like encouraging suicide or law-breaking; the good/bad boundary is fuzzy and subjective, which is exactly what makes alignment verification hard.
Related event: Why verifying AI alignment is harder than verifying capability(3 posts)→
More from AGI Musings
- History repeats: a 1959 mocking poem derided Norbert Wiener's AI warnings — burny_tech · 2026-11-12
- CrewAI Founder: Anti-American-AI FUD Won't Stop Until Volatility Shakes You Out — robleclerc · 2026-11-12
- Kai-Fu Lee wraps 12-hour London blitz promoting 'AI Native' book launch — kaifulee · 2026-11-12
- Do closed AI labs' research agendas diverge? Talent flow acts as a penalty term — stochasticchasm · 2026-11-12
- AI 'doomers' call themselves 'the Cassandra fringe' — critic says the analogy flatters overconfidence — DrTechlash · 2026-11-12
- AI agent traffic reportedly grew 8x in just two months — Dr_Singularity · 2026-11-12