Only 208 of 923 US Jobs Are Covered by AI Benchmarks, Mercor Map Finds
evijit · x · 2026-10-15
Mercor Research released a "Benchmark Map of the US Economy," mapping existing AI benchmarks against ONET's 923 occupations and 18,797 tasks.
Key findings:
- Only 208 of 923 occupations have any task covered by at least 5 benchmark prompts; 6 of 22 job families have none
- Roughly 90% of hours worked in the US correspond to tasks with little or no benchmark coverage
- The 29-benchmark index skews toward the largest labor markets: Computer & Math (29/36) and Business & Financial (33/48) are best covered, while Personal Care (0/31), Production (5/107), and Construction are nearly blank
- Implication: it's very hard to know how good AI models are at the work Americans actually do
The interactive map will be continuously updated as part of Mercor's APEX benchmark effort.
More from AGI Musings
- "Kindness research": keeping old models alive to teach new ones how to care — repligate · 2026-10-15
- Gary Marcus flags AI agent fatigue: managing coding agents burns out engineers — GaryMarcus · 2026-10-15
- Blogger predicts an unprecedented transparency wave as AI makes digitizing everything worthwhile — curious_vii · 2026-10-15
- AI safety researcher defends rigor of METR and Redwood work, explains why labs tout funding — austinc3301 · 2026-10-15
- AI Safety Incentives Debate: Nonprofit Worker Pushes Back on 'Inflating Problems for Funding' Critique — austinc3301 · 2026-10-15
- Schools should teach change, not skills: math community's identity crisis over AI — soumitrashukla9 · 2026-10-15