Study of 2,124 runs: multi-agent dynamic concurrency often hurts coding agents
新智元 · wechat · 2026-10-19
Researchers from Nanjing University and UIUC ran large paired experiments on dynamic concurrency in Codex, Claude Code, and Kimi Code: 354 SWE tasks, 2,124 executions, 11.2B tokens ($20,780).
Key findings
- Unstable gains: clear wins on long-horizon tasks—Claude Code on LoopsBench jumped from 7.1% to 21.4%, Codex on NL2Repo from 34.6% to 57.7%—but short-horizon tasks regressed: on SWE-bench Verified Claude Code fell 83%→59%, Kimi Code 77%→59%
- Cost: concurrency consumed 1.41–3.31× tokens; 14 of 15 agent/benchmark combos took longer on average
- Scheduling differs: Codex delegates earliest (4% into execution), Claude Code sustains the most overlap (52.7%); more agents ≠ more effective parallelism
28 failure modes (4 categories, 13 subcategories)
- Task orchestration (28.5%): pseudo-concurrency, missing owners, load imbalance—e.g., 17 subagents doing investigation only, ending 0/785 tests passed
- Execution governance (29%): a main agent waiting 2 hours on a 40-minute budget while finished subagent work went unintegrated
- Global context management: unclear interface contracts, handover overload, missing global verification
Conclusion: concurrency gains are task-dependent; the real question is converting parallel work into end-to-end benefit. Paper and data are open-sourced.
More from coding & agent
- Claude cut a 2.6-hour video export to 20 minutes — but only after being told to rebuild the server — byhewar · 2026-10-23
- 3 flight-booking rules my AI agent checks that I'd never do by hand — Efistoffeles · 2026-10-23
- OpenJet claims Qwen3.8-27B coding runs 1.9x faster than llama.cpp on one RTX 3090 — JeremyCMorgan · 2026-10-23
- Building an autonomous SEO agent in Hermes with three skills: programmatic pages, Keywords Everywhere, DataForSEO — EXM7777 · 2026-10-23
- Building a sports betting AI agent with onchain execution and human approval by default: architecture and lessons — Dry-Comfortable-2678 · 2026-10-23
- How do you audit AI agent decisions in regulated workflows? A field survey — Ok_Way_1993 · 2026-10-23