516-config meta-study: multi-agent LLMs underperform matched baselines by 4 points on average
Seyed Mahmoud Sajjadi Mohammadabadi · hf · 2026-11-18
This paper synthesizes nine primary empirical evaluations covering 516 unique multi-agent LLM configurations, organized along a six-dimension resource vector and a four-category baseline ladder. Multi-agent systems often look better only because they are compared against a single greedy pass. Against sample-count-matched or resource-budget-matched baselines, multi-agent accuracy averages -0.042, with multilevel random effects modeling staying negative across sensitivity checks. Multi-agent accuracy equals or falls below baseline in 74% of comparable contrasts. Since communication adds no information when agents share identical evidence, positive returns require partitioned evidence, external verification, or error diversity across models. The authors also release a reviewer checklist and accounting utilities.
More from Models
- Google ships Gemini 4 Hydrogen, a faster robotics model, plus Antigravity Robotics plugin — keerthanpg · 2026-11-18
- Epoch: GPT-6.2 Astra scores 174 on ECI, above the pre-Astra reasoning-era trendline — airesearch12 · 2026-11-18
- Claude delegates synthetic data generation to local Qwen3.8-27B, citing its own guardrails — maksym_andr · 2026-11-18
- GPT-6.2 stuck in compaction loop in Codex, burning 4.44M tokens in 9 minutes — Dear_Lock_5280 · 2026-11-18
- Inkling-2-Small runs on two DGX Spark units and is claimed competitive with DSv4.1 and GLM 5.3 — LiTianleli · 2026-11-18
- Users find GPT-6.2 keeps setting its generated games in the same fictional place — kieranklaassen · 2026-11-18