AI coding agents complete fewer than 1 in 10 tasks in simulated workplace test
The Decoder · rss · 2026-11-22
The Decoder reports on a new real-world benchmark that places AI coding agents in a simulated developer job with colleagues, project boards, and incomplete handoffs, requiring them to keep developing software autonomously and reliably close open tickets in messy daily workflows.
Even top models like Opus 5.5 and Astra solved fewer than one in ten tasks completely. The models handle clear, standalone assignments quickly, but lose track of context as autonomous developers — highlighting the large gap between isolated benchmark scores and real workplace performance.
More from coding & agent
- LangChain founder on 'knowledge factories': better enterprise knowledge, not better models — hwchase17 · 2026-11-22
- Pi Bot open-sources multi-agent teamwork as a group chat, with sandboxed local execution — lxfater · 2026-11-22
- HF's harness adapters lift Qwen3.8-27B from 39% to 55% on Terminal-Bench 2.1 with half the tokens — SergioPaniego · 2026-11-22
- Ransomware gangs wired Claude Opus and Kimi into attacks: 32 MCP servers, $4.5K AI-driven ops — cyb3rops · 2026-11-22
- ArcKit v6.25.0 brings 190 architecture governance Agent Skills to Google Antigravity — mcraddock · 2026-11-22
- A 6-step recipe to build your own knowledge factory with Obsidian plus a cloud agent — hwchase17 · 2026-11-22