GDP.xlsx benchmark: 70 real spreadsheet tasks, best frontier agent scores only 38.3%
echen · x · 2026-10-01
A follow-up to GDP.pdf, the new GDP.xlsx benchmark simulates real-world spreadsheet work.
- 70 tasks across 12 domains; the hard part is decoding how a workbook works: which tab is stale, what red cells mean, which comment explains exceptions, which chart Finance trusts, checking termination clauses in Sheet 3.
- The best frontier agent scores 38.3% — reportedly Gemini 4 Argon.
The benchmark highlights implicit context and cross-sheet reasoning, which remain major weaknesses for current agents.
Related event: Surge AI Releases GDP.xlsx Benchmark; Top Agent Scores Just 38.3%(2 posts)→
More from Models
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- Stealth Anthropic model spotted in Devin build as METR chart shows frontier pace up 1.4x — daniel_mac8 · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- Unsloth Q8 crushes default Q3 in a self-hosted test over 331 financial PDFs — No_Cartographer3953 · 2026-11-16
- METR time-horizon chart updated: doubling every 1.7 months since Opus 4.5, 'Model 4' ~38h — ben_j_todd · 2026-11-16