516-config meta-study: multi-agent LLMs underperform matched baselines by 4 points on average

Seyed Mahmoud Sajjadi Mohammadabadi · hf · 2026-11-18

This paper synthesizes nine primary empirical evaluations covering 516 unique multi-agent LLM configurations, organized along a six-dimension resource vector and a four-category baseline ladder. Multi-agent systems often look better only because they are compared against a single greedy pass. Against sample-count-matched or resource-budget-matched baselines, multi-agent accuracy averages -0.042, with multilevel random effects modeling staying negative across sensitivity checks. Multi-agent accuracy equals or falls below baseline in 74% of comparable contrasts. Since communication adds no information when agents share identical evidence, positive returns require partitioned evidence, external verification, or error diversity across models. The authors also release a reviewer checklist and accounting utilities.

Original post →

More from Models

Models channel →