Cohere Labs tests 690K+ tools for agents — only 2.6% can complete a job task alone
cohere · x · 2026-09-04
Cohere Labs released the Agentic Task Ecosystem, a new dataset of 690K+ tools built to test whether AI agents can autonomously complete real occupational tasks.
Under the test of whether a single tool could independently complete an occupational task, only 2.6% passed — a stark quantitative counterpoint to sweeping claims about AI already replacing job tasks. The dataset offers researchers a large-scale benchmark for measuring agentic capability in real work contexts.
More from AGI Musings
- Richard Socher: AI could compress a century of scientific breakthroughs into a decade — RichardSocher · 2026-10-21
- Ethan Mollick: AI's work capacity doubled 8 times in 30 months, METR benchmark maxed out — One Useful Thing (Ethan Mollick) · 2026-10-20
- Time for Guardrails on Government Too: Rethinking Who AI Policy Constrains — neil_chilson · 2026-10-20
- Steam needs real quality control on AI slop, not an AI art ban, argues designer — AIandDesign · 2026-10-20
- Why personal AI agents fall into a retention dead zone between dopamine and infrastructure — signulll · 2026-10-20
- Scholar writes AI-assisted book on storytelling values, work nominated for EMNLP best paper — _akpiper · 2026-10-20