ParallACT benchmark: parallel coding agents in shared repos drop from 62% to 16% on conflicting tasks
EliasEskin · x · 2026-11-04
A UT Austin team (Rishabh Mediratta, Lingyun Xiao, with Peter Stone, Joydeep Biswas, Elias Stengel-Eskin) introduces ParallACT, a benchmark for orchestrator-free multi-agent coding in shared codebases:
- Setting: developers independently delegate tasks to a shared repo, making multi-agent coordination unavoidable — individual implementations may each work but conflict when combined.
- Design: complementary tasks (separately useful, must share an interface) and contradictory pairs (both agents edit the same code incompatibly).
- Findings: a single-agent oracle given the combined task set solves most tasks, while parallel agents perform substantially worse. Drops are notable on complementary tasks and sharp on contradictory ones — Claude Code with Claude Opus 4.8 falls from 62% to 16%, with merged code typically satisfying only one spec; the gap persists across six further configurations varying execution order, visibility, awareness, and communication channels.
Paper, code, and data are public.
Related event: ParallACT benchmark: parallel coding agents drop to 16% success(9 posts)→
More from coding & agent
- Open-source 'Palantir for Cybersecurity' maps live exploits and routes fixes to coding agents — Fowe · 2026-11-04
- Matt Shumer opens his agent for SMS trials, giving $150 in free credits to new users — mattshumer_ · 2026-11-04
- Indie dev adds full MCP support to SnapWidget, from signup to payments — hermanschutte · 2026-11-04
- Forrester Predicts 2027: Unreviewed AI Code Will 'Blow Up' a Company, 15% of Developers Job-Hunting — rseroter · 2026-11-04
- Model Vendors Post-Train for Tool Search, Squeezing Out MCP-Layer Workarounds — pvncher · 2026-11-04
- Engineer builds agent that clears years-old code backlog without any mandate — BradPorter_ · 2026-11-04