ICLR reviewer slams AI safety papers as CV padding; critic says Goodharting wrecked the field
JacquesThibs · x · 2026-10-19
ML researcher @kotekjediml complained that most AI safety "papers" in their ICLR review batch are low-quality CV blips between undergrad and a role at places like Anthropic, warning the names will be public. Jacques Thibodeau quoted and added that the optimization pressure produced reward hacking/Goodharting that has messed up the entire AI safety field — a notable inside critique of mass-produced safety research.
More from AGI Musings
- Fields Medalist Hugo Duminil-Copin refuses AI in his own math research: "It gives you 42 but you have zero clue why" — Hesamation · 2026-10-19
- Terence Tao: AI is ushering math into an era of proof abundance — lmoroney · 2026-10-19
- Three fundamental problems with Anthropic's definition of 'public benefit' — Dr_Atoosa · 2026-10-19
- MIT Researchers Stranded Claude on a Virtual Island for 30 Hours — It Started to Play — MIT Tech Review AI · 2026-10-19
- If scaling holds, model training becomes a monopoly with one insurmountable leader — abhiadesai · 2026-10-19
- Viral theory claims RSI-capable models can't be open-sourced, DeepSeek dates are frozen snapshots — teortaxesTex · 2026-10-19