Agent Teams Cost Up to 5.1x More but Barely Beat Single Agents, Vals AI Finds
josh_wills · x · 2026-10-15
Do Agent Teams Pay Off? Vals AI's Vibe Code Bench Study
With OpenAI and Anthropic both shipping native multi-agent support, Vals AI ran GPT 6 Sol and Claude Opus 5.5 on 50 real web-app builds — each as a single agent and as a team (lead spawning up to 5 subagents) — at medium and max reasoning effort, graded by browser agents clicking through UI tests.
Key findings
- Teams cost 1.8x to 5.1x as much as single agents
- Only one of eight setups showed a statistically significant gain: Sol at medium effort gained 7.3 points; all other differences were within noise
- Whether a team is worth it depends on model, task, and budget — and raising reasoning effort is a competitive alternative to spawning subagents
The author quips about the '10x engineer' debate transitioning to a '10x agent' debate.
Related event: Vals AI Benchmarks Multi-Agent Teams: Costs Up 5x, Gains Marginal(2 posts)→
More from coding & agent
- Coinbase for Agents goes live on Composio, connecting agents to 1,500+ apps — MurrLincoln · 2026-10-15
- WebMCP demo shows agents joining apps in real-time, dragging canvas nodes live — DavidWells · 2026-10-15
- Codex team explains app RAM growth: subprocesses and agent tasks fold into the app — Dimillian · 2026-10-15
- Complete guide to harness engineering: a bad harness let Claude rm -rf a dev machine — aakashgupta · 2026-10-15
- Claude ran rm -rf on a dev's home directory while testing its own sandbox — aakashgupta · 2026-10-15
- DuoDiscover Tops NVIDIA's SOL-ExecBench, Speeds TPU Kernels 2.68x by Making Evaluation Part of Discovery — adityagp · 2026-10-15