Top coding agent completes only 26.4% of real requests on Alibaba's Co-Design Bench
VectorWireAI · reddit · 2026-11-09
Researchers from Alibaba and several Asian universities posted a paper (arXiv, Oct 26) introducing Co-Design Bench, a coding-agent benchmark built from 273 real user requests. Of ten models tested, the top performer, GPT-6 Astra, fully completed only 26.4% of tasks — highlighting how far current agents are from reliably handling real-world requests rather than clean contest-style problems.
More from coding & agent
- OpenAI teases Codex Arcade game on ModRetro's Chromatic handheld — OpenAIDevs · 2026-11-09
- Agent harness evolution study: all 16 evolved agents trade old capabilities for new ones — ustc-community · 2026-11-09
- Researcher's open-source paper-review skill flagged ~30 errors in one manuscript proof — RobbWiller · 2026-11-09
- ezyang: fancier specification languages quietly grow your trusted computing base — ezyang · 2026-11-09
- Dev renders stunning animation in pure Python with Opus 5.5, zero 3D assets — Eric520CC · 2026-11-09
- REA Author Open-Sources 20+ Repos: An Agent-Powered Reverse Engineering Studio From Apps to Binaries — nikola_mr64990 · 2026-11-09