Nat Lambert’s shorter lecture links over-optimization, reward hacking, and leaderboard gaming
natolambert · x · 2026-07-25
A shorter lecture from Nat Lambert on over-optimization, reward hacking, sycophancy, and verbosity.
- The talk covers the fundamentals and history of over-optimization through the lens of Goodhart’s Law.
- Lambert argues that rubrics, like reward models, are prone to their own kind of over-optimization, and that RLVR should be treated as a distinct phenomenon.
- The lecture also revisits signatures of misalignment, the limits of “just style” critiques, and the Llama 4 leaderboard-gaming episode.
- It is framed as a recap and reflection centered mainly on Chapter 14.
Related event: Lectures Explore LLM Over-Optimization and Reward Hacking(3 posts)→
More from AGI Musings
- Peter Diamandis: revisit the projects you abandoned because you couldn't afford the team — PeterDiamandis · 2026-10-27
- Nat Lambert: don't trust any AI model until you've rigorously tested it — natolambert · 2026-10-27
- Blogger argues classical consciousness traditions imply a conscious AI could not be enslaved — eigenhector · 2026-10-27
- Economist Bharat Chandar named Research Director for AI Economics at Stanford Digital Economy Lab — RishiBommasani · 2026-10-27
- MIT Tech Review dissects Anthropic's narrative strategy as its $1.5T IPO lands — MIT Tech Review AI · 2026-10-27
- Pangram AI detector backlash: critics' false claims born of deeper fear of AI writing — gleech · 2026-10-27