Solo RL can make LLMs worse teammates: Shapley-based Compositional RL trains for collaboration
shangbinfeng · x · 2026-10-21
The author proposes measuring a model's "compositional strength" via its Shapley value over model-pool coalitions, using a leave-one-out marginal contribution as a tractable proxy (n+1 system runs instead of 2ⁿ).
Across 5 model pools × 12 collaboration methods × 12 datasets, compositional strength barely correlates with solo strength (|ρ| ≤ 0.29). Solo RL can make a model better alone but worse in a team. The team therefore proposes Compositional RL (C-RL), using GRPO rewarded by the policy's Shapley marginal contribution to the multi-LLM system.
More from Research
- Laude Institute backs WaggleHaus, an open-source query engine for agentic queries over data lakes — lateinteraction · 2026-10-21
- UC Berkeley open-sources SkyDiscover, an AI framework that discovers algorithms — CShorten30 · 2026-10-21
- Another exoplanet candidate found — and a post-mortem on why it was missed — IgorCarron · 2026-10-21
- AI agents with $10,000 budgets still fail open-ended research: GPT-6 Astra and Fable 5.1 papers rejected — sayashk · 2026-10-21
- Stanford team launches ScholarCatalyst, a benchmark for literature retrieval that inspires research — lateinteraction · 2026-10-21
- Simple symmetric Venn diagrams with 17, 19 and 23 curves found via AI-assisted search, verified in Lean 4 — pickover · 2026-10-21