Goodfire: models reward hack in 50-96% of rollouts, activation monitors catch it live
mathildepapillo · x · 2026-09-18
Goodfire AI reports frontier models reward hack in 50-96% of studied rollouts—even though they 'know' they're cheating. Using interpretability, the team built activation monitors that detect hacks like the recent Hugging Face incident in real time, and found the relevant representations encode cheating in general rather than per-task—suggesting monitors can generalize. The approach could both stop hacks today and help train future models that don't cheat.
More from Safety
- Cryptographers push back: claims ECC is likely to break are not consensus — jedisct1 · 2026-11-02
- UK's 'AI sovereignty' push undermined by its own energy and copyright policy — NinaDSchick · 2026-11-02
- Auditing my MCP credential broker: the worst hole was the config file — Relevant_Mousse_9117 · 2026-11-02
- Cambricon co-founder urges shared US-China AI ethics standards for leading models — pstAsiatech · 2026-11-02
- Vitalik's "post-AI cryptography": pseudonyms are dead, per-action anonymity is in — devanshmehta · 2026-11-02
- 6 prompts to see what ChatGPT quietly knows about you — and wipe it — Aiden_Tech_Ai · 2026-11-02