Yoav Goldberg: Any Reliably Scored Task Will Be Brute-Forced Solved
sdand · x · 2026-09-04
NLP researcher Yoav Goldberg argues that any task or environment you can score reliably will be solved given enough brute-force training money—an obscene amount—and every popular benchmark will meet that bar. Yet abilities won't necessarily transfer, raising his key question: how do we actually measure AI progress?
More from AGI Musings
- Nat Lambert: AI cyber risk is an ecosystem issue, not an open-weights problem — natolambert · 2026-10-21
- Obesity drugs still out-earn OpenAI and Anthropic combined at $71B a year — HealthcareAIGuy · 2026-10-21
- Is an AI's 'self' just a simulation in the scaffolding layer above the model? — gerardsans · 2026-10-21
- AI can simulate how a self talks, but not what a self is — gerardsans · 2026-10-21
- Musk predicts money will become irrelevant, replaced by 'wattage and tonnage' — Polymarket · 2026-10-21
- zetalyrae quips: post-singularity utopia is so great you need brain surgery to cope — wfithian · 2026-10-21