Mercor: only 208 of 923 US occupations are covered by existing AI benchmarks
dan_fried · x · 2026-10-20
Mercor Research released the "Benchmark Map of the US Economy," an interactive breakdown of which economic tasks are covered by AI benchmarks, alongside an arXiv paper on how agent development reflects real-world work.
Key data:
- Only 208 of 923 US occupations are represented by existing AI benchmarks; 90% of hours worked in the US lack a relevant benchmark.
- The study analyzed 43 benchmarks and 72,342 tasks, mapping them to 1,016 US occupations, revealing that agent development is programming-centric and badly mismatched with where human labor and economic value concentrate.
- Author Zora Wang notes this distributional gap persists 10 months after her team's earlier mapping study.
- The paper proposes three principles for better benchmarks: coverage, realism, and granular evaluation.
More from AGI Musings
- AI copy converges to the median; unique expertise becomes the most valuable asset — daluoseo · 2026-10-20
- Post-AGI 'meaning' debates ignore neurotech: why can't we just turn off dread? — MWCvitkovic · 2026-10-20
- Researcher jokes AI field is 'rudderless' after a year without Altman's updated forecasts — generativist · 2026-10-20
- Lessons from Musk, Zuckerberg and Napoleon: small cracked teams, big risks, no hesitation — davidsenra · 2026-10-20
- Against victim-blaming: 'Don't rape someone who can't defend themselves' should be the message — Introvert_Sci · 2026-10-20
- Kroger cuts 3,000 jobs leaning into AI; companies pay for headcount, rules, power — YvesMulkers · 2026-10-20