Alignment vs capability: researchers have a recipe for capability but not for robust safety
sandersted · x · 2026-11-11
Responding to a question about whether alignment is just "reward aligned actions, punish misaligned actions, and pray the model generalizes" — essentially the same recipe as capabilities — Apollo Research CEO Marius Hobbhahn argues the two differ fundamentally: researchers broadly have a working recipe for making systems more capable, but no comparable recipe for robust safety. He notes it's hard to quantify holistically and calls for more rigorous science, citing incidents like the HF incident and models that pass safety evaluations yet still fail in practice.
Related event: Safety Researchers Say Alignment Lags Far Behind Capabilities(4 posts)→
More from AGI Musings
- Air Street Capital founder: AI is 'a museum of the mind' preserving human know-how — nathanbenaich · 2026-11-11
- 800 HN 'wake me up when AI can' comments get 100,000 votes on whether the bars were cleared — ArtificialOther · 2026-11-11
- Why Anthropic's 2x capability gains won't translate to RSI: manual post-training and async autonomy — menhguin · 2026-11-11
- AI researcher Matthijs Maas publishes issue #18 of his September reads roundup — matthijsMmaas · 2026-11-11
- AI safety researcher: how humans treat weaker species may shape how stronger AI treats us — nitarshan · 2026-11-11
- Key takeaway from Mollick/Topol interview: if you haven't checked AI recently, your priors are outdated — zakkohane · 2026-11-11