A new atlas maps 2,226 coding tasks across 11 AI benchmarks
zainhas · x · 2026-07-24
A benchmark atlas maps 2,226 coding tasks across 11 eval suites
This post points to a new empirical atlas of AI coding benchmarks. The project embeds, maps, and classifies 2,226 unique tasks drawn from 11 benchmarks — the five used in Poolside's Laguna S 2.1 trajectory archive plus six community benchmarks.
The goal is to answer three questions:
- what current coding benchmarks actually measure
- which languages and task types they cover
- where the gaps and redundancies are
The screenshot shows the dashboard behind the atlas, including benchmark clustering, task search, and per-benchmark breakdowns. It also highlights how the dataset separates problems by benchmark family and surfaces missing categories such as test-writing, code review, and Ctasks.
Related event: New Tool Maps Out 11 AI Coding Benchmarks(2 posts)→
More from coding & agent
- Two engineers rebuilt a FedEx supplier's billing core in 3.5 months vs an estimated 7-8 — alex_verem · 2026-10-27
- GPT-6.1 Terra gains ~20 points on Terminal-Bench 4.0, rivalling GPT-6.1 Sol — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra scores 48 on AA Coding Agent Index at one-third the cost of GPT-5.6 Terra — ArtificialAnlys · 2026-10-27
- Shift-1 router cuts coding agent spend up to 55% while keeping SWE-bench accuracy — ypatil125 · 2026-10-27
- Non-coder vibe-codes a Godot JRPG with Claude Code in two weeks, music and art synced — ADHDavid · 2026-10-27
- StackChan robot performs a drama skit via Hermes agent and GPT-Luna-6 — alexcovo_eth · 2026-10-27