Alibaba's Co-Design Bench: top model fully solves only 26.4% of real coding tasks
The Decoder · rss · 2026-11-08
The Decoder covers Alibaba's new coding-agent benchmark, Co-Design Bench, built from real user requests that tripped up AI agents in practice. Even the top performer, GPT-6 Astra, fully solves just 26.4% of tasks — meaning nearly three out of four still need human cleanup — while the same models score above 99% on older benchmarks, underscoring how far evals lag reality.
More from coding & agent
- AFK Pilot extension brings Grok, Claude, Codex, DeepSeek and more into one VS Code window — PawelHuryn · 2026-11-08
- 17 open-source offensive AI security frameworks: Cairn hits 4.9k stars — evilsocket · 2026-11-08
- 'Find all the implicit state machines in my app' becomes a favorite AI prompt — rachittshah · 2026-11-08
- Claude Code sessions can message each other — demo shows agent-to-agent collaboration — dejanseo · 2026-11-08
- Qwen builds its own LLM: trains a 100k-parameter model in your browser, and narrates it — victormustar · 2026-11-08
- Dev open-sources Codemm: turn your own coding agent into a step-by-step tutor — gael55x · 2026-11-08