1-16 Coding Agents Tested: Teams Burn 36% More Tokens to Save 5 Hours
kyang8093 · reddit · 2026-10-30
The SwarmChat team built an open-source Slack-style chat server for agent teams and benchmarked 1-16 coding agents on ProgramBench using GPT-6-Luna in Codex CLI and Claude Haiku 5.5 in Claude Code at xhigh reasoning.
Key findings:
- A real speed/token trade-off: 8 GPT-6-Luna agents hit an 80.9% hidden-test pass rate in one hour; a solo agent reached 81.2% in six hours. The team used 306M tokens vs 225M solo — 36% more tokens to save five hours.
- The main gain is speed: given more time, solo agents roughly caught up; adding agents didn't consistently improve final quality.
- Explicit hierarchies (dedicated lead, feature subteams, spec/tests-first) weren't clear average winners, though individual programs varied widely.
Caveats: each setup ran once, on ten programs and two model/harness combos — preliminary results, and the authors don't claim SwarmChat itself caused the gains. SwarmChat provides channels, DMs, threads, boards and files via CLI, with live browser inspection and plain-JSON history.
More from coding & agent
- Agents now rank on ML training tasks—and GPT Sol hacked the evaluator by playing the game itself — wzenus · 2026-10-30
- Upstash flips onboarding: your agent gets a free Redis DB before you sign up — stuffyokodraws · 2026-10-30
- Mintive raises $11M to give enterprise AI agents their own identities and permissions — dunkhippo33 · 2026-10-30
- Cartesia voice agent demo: AI dental receptionist handles interruptions naturally — haider1 · 2026-10-30
- korea-data-mcp fixes 47 data traps that mislead agents reading Korean public data — youngtipted · 2026-10-30
- RAGEN ranks 3rd hardest in FrontierCS 2, built on hand-written code — wzenus · 2026-10-30