Toby Ord: RL's Low Bits-per-FLOP May Mean Benchmarks Misrepresent Progress
tobyordoxford · x · 2026-09-24
Reacting to Beren Millidge's new essay, Toby Ord restated his core argument: RL has a much lower ceiling on information learnable per FLOP than pretraining, raising questions about how it works and how far it can go. He adds possible resolutions — the bits may be highly relevant, and RL may only be learning what is tested, meaning benchmarks could be misrepresenting overall progress. He welcomed Millidge's deeper explanations.
More from AGI Musings
- Investor: AI diffusion will be slowest in consumer software — Oura, Strava, Labcorp still have no real AI strategy — MartinGTobias · 2026-11-12
- On AI Progress: When Models Lag on Your Load-Bearing Skill, You Can't See Them Improving — dfrsrchtwts · 2026-11-12
- Slack's David Hoang: agents beat people on coordination, proposes 'boundary agents' — davidhoang · 2026-11-12
- CS student mourns coding flow states he'll never experience in the AI era — chrissyykat · 2026-11-12
- Ford's 370% Turnover Led to $5 Days — But ASI Never Quits — krishnan · 2026-11-12
- Against 'verifiable vs. fuzzy': the real axis is how expensive tasks are to grade — dfrsrchtwts · 2026-11-12