Thom Wolf shares a shorter lecture on over-optimization, reward hacking and sycophancy
Thom_Wolf · x · 2026-07-26
A shorter lecture on over-optimization, reward hacking, sycophancy, and verbosity.
Thom Wolf says the talk is mostly about fundamentals, history, and reflections. While recording it, he notes that rubrics may themselves be prone to over-optimization in the same way reward models are, and that RLVR is its own distinct thing.
The lecture is structured around:
- over-optimization and Goodhart’s law
- the foundations of reward hacking
- sycophancy and verbosity
- why evaluation rubrics can be gamed
Related event: Lectures Explore LLM Over-Optimization and Reward Hacking(3 posts)→
More from Research
- New review: how prefrontal cortex dynamics turn world models into flexible behavior — AnnaCiaunica · 2026-10-27
- LIBERO-MAX: Robot Policies Drop 11–26 Points When the World Changes Mid-Task — drmapavone · 2026-10-27
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- One BBQ Example Pushes LLM Fairness Scores Past 90%, EMNLP Paper Questions Benchmarks — radamihalcea · 2026-10-27
- Real2Sem2Real turns one phone video into 2,000+ robot demos, matching 150 teleop demos — Scobleizer · 2026-10-27
- Understanding MuonH (Hyperball) through weight norm geometry of scale-invariant networks — jasondeanlee · 2026-10-27