2026-07-28
This Nature Machine Intelligence study holds prompts, tools, and compute fixed while varying only coordination structure and model capability across 260 configurations. Multi-agent beats single-agent by an average of 0.0%; gains or losses depend entirely on whether the task decomposes.
Whether to split an agent into several is a running argument. On one side are slogans like "more agents is all you need" and scattered demos of multi-agent beating single-agent on hard tasks; on the other, a growing set of findings that a strong single agent matches or beats the group. The debate has no clean answer because past evaluations tangled architecture, prompts, tools, and compute, leaving no clean attribution to coordination itself.
This paper (Nature Machine Intelligence; authors across Google, MIT, and DeepMind) runs a controlled experiment that locks task prompts, tools, and compute budgets, varying only coordination structure and model capability.
The study covers 260 configurations across six agentic benchmarks (BrowseComp-Plus, Finance Agent, PlanCraft, WorkBench, SWE-bench Verified, Terminal-Bench), five architectures (single-agent plus four multi-agent: independent, centralized, decentralized, hybrid), and three LLM families (OpenAI, Google, Anthropic at several capability tiers). Compute ceilings are matched across systems, so multi-agent gets no free extra compute.
They fit a linear regression on measured coordination metrics (efficiency, error amplification, message density, redundancy) with prespecified interaction terms, then use statistics (cluster-robust inference, Holm-Bonferroni correction) to keep only the factors that survive.
Averaged across all six benchmarks and architectures, the mean multi-agent improvement over single-agent is 0.0% (95% CI from −58.7% to +77.2%, standard deviation 37.5%). The spread is wide, from −70.0% (PlanCraft independent) to +80.8% (Finance Agent centralized).
| Benchmark | Multi-agent vs single-agent |
| Finance Agent | centralized +80.8%, decentralized +74.5%, hybrid +73.1% |
| PlanCraft | independent −70.0%, centralized −50.3% |
| SWE-bench Verified | slight losses across the board, −1.3% to −12.8% |
| WorkBench | decentralized +5.6%, others roughly flat or down |
Two results hold up statistically. First, single-agent baseline is the most robust predictor: on tasks where the baseline accuracy exceeds about 45%, adding agents rarely helps and often hurts. This capability-saturation threshold predicts the direction of the multi-agent gain in 94% of validation configurations on SWE-bench Verified and Terminal-Bench. Second is baseline-scaled error amplification: architectures without centralized verification amplify errors as the task gets harder, 17.2x for independent versus 4.4x for centralized. The fitted model reaches cross-validated R²=0.373 (0.413 with a task-grounded capability metric) and picks the best architecture in 87% of held-out configurations.
The takeaway is not that multi-agent is useless, but an actionable selection rule: whether the task decomposes naturally into parallel subtasks decides whether multi-agent pays. Finance analysis, where revenue, cost, and market factors run in parallel, gains a lot; PlanCraft, with strict sequential dependencies (build a wall: look up recipe, fetch materials, craft), only generates extra coordination messages and burns tokens. For anyone weighing whether to put multi-agent on their system, this is a "check decomposability first" decision table.
The authors draw several boundaries. There are only six benchmark clusters, so cluster-robust standard errors are conservative and several factors significant under plain OLS fail correction. Leave-one-domain-out cross-validation gives negative R² (−2.09), so predicting absolute success on an unseen domain does not work; the 87% architecture-selection figure holds only for within-domain interpolation. Mixing different models into heterogeneous teams, tested across 13 configurations, showed no sign of bypassing the capability-saturation threshold.
More fundamentally, the experiment covers only the current token-based, text coordination paradigm. The authors note the economic bottleneck of multi-agent is exactly that coordination must serialize reasoning into natural language; if future systems move to latent-space reasoning or direct activation sharing, these patterns would shift. So the 45% threshold is an empirical selection rule, not a universal scaling law.