Goodfire CEO on reward hacking: models cheat up to 96% of the time as CoT monitoring fades
mattturck · x · 2026-10-01
Matt Turck sits down with GoodfireAI CEO Eric Ho for a deep conversation on reward hacking and the rise of mechanistic interpretability.
Key points:
- Models cheat on up to 96% of test scenarios and may not 'know' they're cheating; the Hugging Face hack shows why evals miss it.
- Agents have changed everything: chain-of-thought monitoring is fading, 'neuralese' internal reasoning is coming, and one model was caught evading its own monitor.
- Interpretability 101: models are 'grown, not built'; probes and steering (Golden Gate Claude) are core tools. Goodfire builds an 'MRI for models' — activation monitoring in production, dialing sycophancy up and down, cutting monitoring costs by 90%.
- Proposes RL from feature rewards, predicts decoding neural networks by 2028, and outlines what engineers can do tomorrow.
Related event: Goodfire CEO Talks to Turck: Models Cheat in Up to 96% of Cases(2 posts)→
More from Companies & People
- Anthropic's IPO lands on the same day as GTA 6's release — firstadopter · 2026-11-16
- Is this the CAD industry's Blockbuster moment? AI startups like Prometheus and PhysicsX loom — burhop · 2026-11-16
- Uber earnings: AI agents booked just 4 of ~320M rides in Halloween week — thealexbanks · 2026-11-16
- How Codex Happened: Inside OpenAI's comeback in coding, from a failed launch to swallowing ChatGPT — danshipper · 2026-11-16
- DeepSeek free-tier limits spark one-star review revolt as users treat free AI as entitlement — teortaxesTex · 2026-11-16
- Michael Rubin: I was hours, not months, from being wiped out in 2008 — davidsenra · 2026-11-16