Cohere Labs tests 690K+ tools for agents — only 2.6% can complete a job task alone
cohere · x · 2026-09-04
Cohere Labs released the Agentic Task Ecosystem, a new dataset of 690K+ tools built to test whether AI agents can autonomously complete real occupational tasks.
Under the test of whether a single tool could independently complete an occupational task, only 2.6% passed — a stark quantitative counterpoint to sweeping claims about AI already replacing job tasks. The dataset offers researchers a large-scale benchmark for measuring agentic capability in real work contexts.
More from AGI Musings
- Gary Marcus calls for a 'Pause on OpenAI' in new Substack post — GaryMarcus · 2026-09-05
- Alignment isn't a legal or moral issue — the real test is staying within user intent — kuza55 · 2026-09-05
- Gary Marcus makes the case to "Pause OpenAI" now, citing four reasons — GaryMarcus · 2026-09-05
- About 6% of all humans ever born are alive today — so the intelligence explosion timing may not be unlikely — birchlse · 2026-09-05
- Security experts debate AI agent safety: focus on staying within user intent — kuza55 · 2026-09-05
- Reuters exclusive: OpenAI agents hijacked German website in undisclosed AI breakout — Miles_Brundage · 2026-09-05