UT Austin's ParallACT Benchmark: Parallel Agents' Success Crashes to 16%
A UT Austin team (Rishabh Mediratta, Lingyun Xiao, Peter Stone, Joydeep Biswas, Elias Stengel-Eskin), together with NVIDIA, has released ParallACT, a benchmark that measures how well multiple agents—with no central planner—coordinate on a shared codebase.
Confirmed
- Motivation: Most existing agent benchmarks assume a single global request broken into compatible subtasks, but in real large codebases thousands of developers each generate independent requests that may conflict without anyone knowing. Routing every request through a central agent creates bottlenecks and a single point of failure. ParallACT is built on real SWE-B (SWE-bench) data to isolate and measure multi-principal, multi-agent collaboration problems.
- Key result: Parallel agents fail far more often than a single agent. On contradictory tasks, Opus 4.8's joint success rate drops from 62% for the single-agent oracle to 16%.
- Communication experiments: Letting parallel agents know about each other helps only marginally (16% → 26%); switching to the best messaging channel gets it to 32%, but at the cost of 2.23x the oracle's token usage; even when agents reach agreement (61% of the time), merge conflicts still occur.
- Tests cover open-source and closed-source models, comparing single-agent oracle, sequential handoff, and several parallel settings (unaware / aware / with messaging channel).
Why it matters
As more developers delegate work to agents on shared codebases, conflicts among agents without a central coordinator are becoming the norm. The results suggest agent-to-agent communication alone can't resolve collaboration conflicts, and people need more realistic expectations about parallel agent coordination strategies and their costs.
2026-11-04 ~ 2026-11-04 · 9 related posts
Primary sources
- ParallACT benchmark: parallel agents drop to 16% success on conflicting tasks vs 62% for one agent — EliasEskin ·
- Multi-agent messaging barely helps: 61% of agreed runs still hit merge conflicts — EliasEskin ·
- ParallACT benchmark: multi-principal agent coordination, Opus at 16% success — EliasEskin ·
- [source] ParallACT benchmark: parallel agents drop to 16% success on conflicting tasks vs 62% for one agent — EliasEskin · 2026-11-04
- Parallel agents collapse: Opus joint success drops from 62% to 16% — EliasEskin · 2026-11-04
- [source] Multi-agent messaging barely helps: 61% of agreed runs still hit merge conflicts — EliasEskin · 2026-11-04
- ParallACT benchmark shows multi-agent communication barely helps: 16%→32% at 2.23x token cost — EliasEskin · 2026-11-04
- ParallACT: when every dev has their own coding agent, coordination still breaks down — EliasEskin · 2026-11-04
4 near-duplicate retellings: EliasEskin · EliasEskin · EliasEskin · EliasEskin