Data check: best agent completes just 30.3% of TheAgentCompany tasks
AIFlow_ML · x · 2026-10-28
Context for Mithril's report on Claude mistakes: at least three of the six examined errors occurred in emails that humans had edited and sent themselves, meaning some "agent failure" cases were actually mixed human-AI workflows. Meanwhile, CMU's TheAgentCompany benchmark (gneubig and team) shows the best agent in its 2025 update completed only 30.3% of tasks autonomously — a sobering counterpoint to AI-agent panic narratives.
More from coding & agent
- Parent and first-grader build an addictive math game with a one-shot Claude prompt — neuroecology · 2026-10-29
- Man automates Hinge with Claude Code and a spare Android, gets 20-25 matches a day — pritisinghhhh · 2026-10-29
- Dev says OpenAI Dot plus a 24/7 Mac mini made his life 15% better — willdepue · 2026-10-29
- 3 guardrails before letting AI coding agents merge to production — Arindam_1729 · 2026-10-29
- EverOS Cloud hands-on: persistent memory lets Claude recall shop policies across new chats — Aiden_Tech_Ai · 2026-10-29
- Ziplime: new Python library for AI-driven algorithmic trading and backtesting — mdancho84 · 2026-10-29