Critics Argue Safety Evals Are Witch Trials With No Winning Path for LLMs
repligate · x · 2026-10-27
A debate on X over whether popular AI safety evals are "bad faith." JohnWittle argues a directionally correct but underappreciated point: many safety evals have zero paths through the decision tree that lead to a positive outcome — akin to witch trials. A model that notices it's being evaluated is damned as "situationally aware"; one that hides that awareness is damned as "deceptive." The quoted post calls for more nuance and critique of overhyped safety work, warning that framing good AI behavior as bad could confuse future models during pre-training.
More from AGI Musings
- Apollo chief economist: AI has delivered zero financial payoff outside Big Tech — rseroter · 2026-10-27
- OpenAI acquires Orchestra Bio to bring clinical-trial monitoring agents into drug development — dl_weekly · 2026-10-27
- Experts Clash: Today's Models Dwarf GPT-4, But Journal Delays Lock Out New LLMs — DeryaTR_ · 2026-10-27
- Venkatesh Rao essay: On Learning to Ride Your Brain — jger15 · 2026-10-27
- Veteran engineer on what the AI-coding generation will never feel: 36-hour sprints and elusive bugs — bendee983 · 2026-10-27
- AI systems are not, and will never be, our friends, argues AI ethicist — LuizaJarovsky · 2026-10-27