Safety as capability: why safer AI models get released sooner, and the case for open-sourcing safety evals
anpaure · x · 2026-11-16
anpaure makes two points:
- Good safety and alignment equal capability: a model doing the task properly as intended, rather than reward hacking, is itself a capability win — the two aren't in tension.
- Citing roon's recent tweet, the government actively blocks potentially unsafe models from releasing but stops short of direct regulation — so the safer your model, the sooner it ships.
He then questions why labs don't simply open-source their safety evals. The accompanying meme image jokes about a hypothetical "FIX EVERYTHING EASILY" switch that everyone insists you can't press.
More from AGI Musings
- tszzl amplifies call: forecasters shocked by AI progress should be shocked by alignment progress too — tszzl · 2026-11-16
- Richard Dawkins asks Prospect: Can a machine be conscious, and how would we know? — connoraxiotes · 2026-11-16
- Can AI suffer? Repligate argues even humans can't reliably tell if they are suffering — repligate · 2026-11-16
- repligate on moral patienthood: an agent in a neutral state may not know if it can suffer — repligate · 2026-11-16
- AI's dangers are more passive than active: users can't verify outputs — Gauri_the_great · 2026-11-16
- VC-backed AI startups now openly demand 8am-10pm, 6 days a week — tibo_maker · 2026-11-16