Test-time agent communication matches 4x compute, and cracks tasks no single agent can solve
DimitrisPapail · x · 2026-09-22
A new arXiv paper, Scaling Discovery through Test-Time Communication (Jongho Park, Akshay Krishnamurthy, Dimitris Papailiopoulos, et al.), systematically studies whether multi-agent test-time communication beats independent parallel attempts:
- Core finding: On ARC-AGI-3, team@k (k communicating agents with no predefined roles, sharing a directory) matches the success rate of 4k independent agents, and the advantage grows with k — gains compound with scale.
- Beyond efficiency: tasks no single agent can solve can be reliably solved by a communicating team.
- Transfers to research tasks: on polyomino packing, communicating agents beat best@k and exceed the prior best-known score; on MNIST classifier compression, a team of four produced a 1,957-byte classifier at 99.4% test accuracy, beating both the best human and best single-agent solutions.
- Caveats: with limited compute or a clear measurable objective, independent agents may win.
The takeaway: science advances through collaboration, but existing agentic systems barely use it — test-time communication is a direction worth scaling.
Related event: Test-Time Communication: Team-of-5 Agents Matches Best-of-33(25 posts)→
More from Research
- 50+ NeurIPS 2025 papers found containing AI hallucinations that slipped past peer review — CSProfKGD · 2026-11-12
- Indie researcher pauses RNN training over environment bugs, shifts to FunSearch-style tasks — cephaloform · 2026-11-12
- Self-supervised visual similarity partially explains word generalization in humans and machines — Patricio M. Vera · 2026-11-12
- P2Echo: one text-prompted model segments multi-view echocardiography with high accuracy and low cost — UBC-V · 2026-11-12
- I Fixed Alamo Drafthouse's Search With Query Logs Alone: +18.4% MRR, No Model Training — CShorten30 · 2026-11-12
- NYU lab open-sources Outerloop: autonomous research agents that open PRs only when benchmarks improve — mengyer · 2026-11-12