AI safety veteran: alignment isn't a property of the model but a dynamic state
tdietterich · x · 2026-09-21
AI safety researcher Thomas Dietterich published a long thread drawing on Nancy Leveson's Engineering a Safer World to argue that safety—or "alignment" in today's parlance—is not a property of automation like self-driving cars or AI agents, but a dynamic property that must be maintained through active control.
Key points:
- Failure modes and environmental changes exhibit unbounded variety; only simple systems in narrow, static settings are exceptions.
- Testing agents that are supposed to reason about novel failures is extremely hard: the best tool is adversarial challenge in simulation, but adversaries operate in a closed move space with no completeness guarantee, and simulations can't be validated in life-threatening scenarios.
- Waymo's "autonomous" vehicles still rely on human supervisor teams, and recent AI cybersecurity incidents at OpenAI and Google show today's AI systems are no different.
Related event: Alignment Community Debates Whether Dynamic Safety Equals AI Alignment(6 posts)→
More from AGI Musings
- PowerShell creator Jeffrey Snover at Ignite: AI revolution is an extinction event for those who don't adapt — dfinke · 2026-11-12
- Worst AI use: signaling you don't care about things you should — Afinetheorem · 2026-11-12
- Stanford HAI blueprint: 56 US agencies report 3,611 AI use cases, evaluation lags — StanfordHAI · 2026-11-12
- Fleuret amplifies debate: AI can't slash wet-lab experiments in unmapped biology — francoisfleuret · 2026-11-12
- Recursive bets on open-endedness: agents running massive parallel experiments like scientist communities — agihouse_org · 2026-11-12
- Recursive's Tim Shi maps recursive self-improvement: from learning data to discovering architectures — agihouse_org · 2026-11-12