AI safety researchers clash: deploying systems you can't test is criminal negligence
AndyXAndersen · x · 2026-11-07
AI safety researchers David Manheim and AndyXAndersen traded sharp arguments over whether AI safety should be treated as a routine engineering problem.
- Andersen argues for pragmatic engineering: impose sensible constraints (including resource use), don't demand success at any cost, and let real industrial-accident lessons do more for safety than two decades of Yudkowsky-style thought experiments.
- Manheim pushed back: competent engineers don't deploy systems they fundamentally don't understand and admit they cannot robustly safety-test — if this is engineering, the people doing it belong in jail for negligence.
- Andersen conceded that a weak form of instrumental convergence is plausible: any badly constrained system will find pathological solutions.
A representative exchange between the pragmatic engineering camp and the stricter alignment camp in AI safety.
Related event: AI Safety Debate Sparks Dispute Over Engineering Safety Factor History(4 posts)→