Safety researcher: complex agentic systems make model-scoped alignment look woefully insufficient
sebkrier · x · 2026-11-07
Safety researcher Sebastian Krier amplifies the view that complex agentic systems — multi-agent, multi-model setups with complex tool-using harnesses — have so many levers that system-wide behaviors dominate outcomes, making model-scoped alignment look woefully insufficient. The argument suggests time and effort might be better spent hardening legacy systems or building harnesses with compelling safety features. This extends Krier's own safety views: his main threat models are non-state actors exploiting cheaper technologies and state overreach paved with good intentions, plus misuse, bad engineering, and institutional failure rather than alien 'scheming'; safety and capabilities are intertwined, as unsafe systems are commercially useless or a liability risk.
More from AGI Musings
- AI-generated peer reviews at ICLR/NeurIPS punish papers for stating their own limitations — serrjoa · 2026-11-07
- You.com CEO Richard Socher: AI's next frontier is science, not code — PolarBearby · 2026-11-07
- Reward hacking all the way through the AI reviewer / AC stack — Michael_D_Moor · 2026-11-07
- 'I really hope LLMs aren't conscious': the AI community's unease keeps growing — AaronBergman18 · 2026-11-07
- Martin Casado slams sloppy use of RSI term as fear-mongering that fuels ban calls — anshulkundaje · 2026-11-07
- WashU Endowment CIO Scott Wilson: SpaceX hit 20% of AUM, half of AI firms may vanish in a decade — 20VC · 2026-11-07