Solo RL can make LLMs better alone but worse together, Compositional RL paper finds
shangbinfeng · x · 2026-10-21
shangbinfeng's team introduces Compositional RL (C-RL), challenging the assumption that solo leaderboards predict collaboration skill:
- LLMs are trained and evaluated alone, but compositional strength — measured via Shapley values (leave-one-out, n+1 runs instead of 2ⁿ) across 5 model pools × 12 collaboration methods × 12 datasets — barely tracks solo strength (|ρ| ≤ 0.29)
- Solo RL can make a model better alone yet worse together — one case shows +6.2 alone but -3.6 together
- Fix: C-RL trains a policy with GRPO, rewarded by its Shapley marginal contribution to the multi-LLM system
Bottom line: a strong soloist is not a team player by default — collaboration must be trained for.
Related event: UW and NVIDIA propose Compositional RL for LLM collaboration(4 posts)→
More from Research
- AI agents with $10,000 budgets still fail open-ended research: GPT-6 Astra and Fable 5.1 papers rejected — sayashk · 2026-10-21
- Stanford team launches ScholarCatalyst, a benchmark for literature retrieval that inspires research — lateinteraction · 2026-10-21
- Simple symmetric Venn diagrams with 17, 19 and 23 curves found via AI-assisted search, verified in Lean 4 — pickover · 2026-10-21
- Pathway CEO in Forbes: Transformers Won't Get Us to Superintelligence, Experience-Learning AI Will — schwentker · 2026-10-21
- Laude Institute Backs Stanford's CLMs: Contrastive Language Models for Fast Decision-Making — Azaliamirh · 2026-10-21
- UC Berkeley releases Actor-Gatherer framework decoupling reasoning from context management — adityagp · 2026-10-21