AI safety evaluations can't assure safety, says researcher urging mandatory limitation preambles
DavidSKrueger · x · 2026-11-17
David Krueger argues you can't fully avoid creating the misconception that AI safety evaluations can tell whether an AI is safe — so evaluators must constantly fight it. He proposes that evaluators begin every outward-facing communication with a preamble: 'AI evaluation is an immature field, incapable of providing the kinds of safety assurance expected in other safety-critical industries such as aviation. None of our findings should be understood to bound risks or verify safety properties of any AI system.'
More from Safety
- Microsoft outage hits airlines and banks, sparking global cyber incident fears — mallow610 · 2026-11-17
- Ex-OpenAI employee says external safety assessors got far less access than Altman promised — j_asminewang · 2026-11-17
- Anthropic quietly pitches its own AI safety bill to lawmakers — marigo · 2026-11-17
- Cambridge's David Krueger: AI safety community is too distracted by daily news horse races — DavidSKrueger · 2026-11-17
- Kelsey Tuoc: AI Companies Won't Give Embedded Auditors Real Employee-Level Access — connoraxiotes · 2026-11-17
- DEF CON talk breaks down 6 data-exfiltration vulnerabilities in AI agents — Kyrannio · 2026-11-17