UW & Ai2's s³: multi-agent RL trains LLM swarms to self-organize and beat best-of-N

StellaLisy · x · 2026-11-07

Researchers at the University of Washington and Ai2 released s³ (Simple Scaling of Swarms via Cooperative Multi-agent RL), training LLM swarms with no leader, no prescribed roles, and no turn order.

The idea

Prior multi-agent systems fix structure in advance (orchestrators, debate rounds, planner/critic roles), causing duplicated work, message overload, and unreconciled subagents. s³ bets that learned organization beats hand-designed organization.
N copies of one model each work on their own task copy, coordinating only via messages they choose to send or read, sharing a single team reward — organization emerges from training.

Results

Claimed first method where an asynchronous, role-free LLM swarm beats best-of-N from solo agents trained with the same budget, and keeps that edge when scaled past the training swarm size.
Agents explore independently, share findings, and pick up where teammates left off.

Code is coming soon; paper and interactive demo are live.

Original post →

UW & Ai2's s³: multi-agent RL trains LLM swarms to self-organize and beat best-of-N

More from coding & agent