MAC Bench: 8-agent teams beat solo agents via headcount, not coordination
gregd_nlp · x · 2026-11-20
Bespoke Labs released Multi Agent Coordination Bench (MAC Bench): twelve natively multi-agent long-horizon tasks measuring how well frontier agent teams coordinate, with tasks and a Harbor-based harness open-sourced under Apache-2.0.
Key findings:
- Current frontier agents coordinate poorly;
- A team of eight agents beats a single agent, but the gain comes from having more agents, not from coordination;
- Among Opus 5.5, GPT-6.1 Sol, and Gemini 4 Argon, only Opus shows signs of real gains from coordinating.
Each task runs in three settings — solo, k agents coordinating over a shared message board (2/4/8), and k agents without coordination — separating headcount gains from coordination gains. The repo includes runnable commands to reproduce with configurable agent count, budget, model, and harness (e.g. claude-code).
More from coding & agent
- Apple reportedly building proactive AI agent that acts autonomously on users' behalf — burny_tech · 2026-11-20
- NormGym benchmark shows agents learn from own norm violations but not from others — xuanalogue · 2026-11-20
- Every adopted 'compound engineering' company-wide so its codebase gets easier to maintain — every · 2026-11-20
- Start custom LLMs at 30B: runs on a single GPU and works with LoRA — ziv_ravid · 2026-11-20
- Not every agent task needs a general LLM — specialized small models by task type — ziv_ravid · 2026-11-20
- Before fine-tuning, distill traces into skills in-context to test if training is worth it — ziv_ravid · 2026-11-20