Yoav Goldberg: it's hard to tell real agent ability from benchmark overfitting
yoavgo · x · 2026-09-05
Yoav Goldberg pushes back on evaluating capabilities like "dynamic multi-step tasks in new environments," noting it's hard to distinguish genuine ability from fine-tuning on a wide-but-specific task set. Galyo's framing: reliably-scorable tasks with enough brute-force training get solved, popular benchmarks will all be beaten, abilities may not transfer — genuine progress is better measured by discoveries like solving open problems.
More from AGI Musings
- Nat Lambert: AI cyber risk is an ecosystem issue, not an open-weights problem — natolambert · 2026-10-21
- Obesity drugs still out-earn OpenAI and Anthropic combined at $71B a year — HealthcareAIGuy · 2026-10-21
- Is an AI's 'self' just a simulation in the scaffolding layer above the model? — gerardsans · 2026-10-21
- AI can simulate how a self talks, but not what a self is — gerardsans · 2026-10-21
- Musk predicts money will become irrelevant, replaced by 'wattage and tonnage' — Polymarket · 2026-10-21
- zetalyrae quips: post-singularity utopia is so great you need brain surgery to cope — wfithian · 2026-10-21