Reward Hacks Devastate RL Training but Barely Matter in Evals, Says Engineer
JoshPurtell · x · 2026-11-02
Josh Purtell argues that training data and eval data face fundamentally different demands. Using reward hacking as an example: proper benchmarks have been dismissed as 'flawed' over minor reward hacks on a few tasks, but in his experience this is devastating for training while barely mattering in evaluations at small doses.
He adds that optimization pressure creates requirements on training data that evals simply don't have — being free of those demands doesn't mean getting lazy, it means working harder on other things.
More from Research
- SketchDirector: agentic layout reasoning generates complex scene sketches — GVCLab · 2026-11-02
- Thought experiment: would AI agents in a physics sim discover laws or reverse-engineer the engine? — zxpawanx · 2026-11-02
- HalachaBench sparks debate: is Talmudic Q&A testing knowledge or reasoning? — teortaxesTex · 2026-11-02
- Sebastian Raschka releases free 11-hour course on building reasoning models from scratch — AxSaucedo · 2026-11-02
- Microsoft and Berkeley open-source TokenEconomizer, cutting coding agent costs 18–31% — victor_explore · 2026-11-02
- TLA-Verus uses LLM agents to prove temporal properties of TLA+ specs — tianyin_xu · 2026-11-02