UW × NVIDIA paper: Compositional RL trains a 27B LLM to be the best teammate in multi-model systems
shangbinfeng · x · 2026-10-21
Paper arXiv:2610.17456 (UW × NVIDIA; authors include Yulia Tsvetkov, Jan Kautz, David Acuna) argues that models are evaluated solo but increasingly deployed in routed, debating, ensembled, or co-trained systems—and standalone benchmarks poorly predict system value.
Key points:
- Formalizes "compositional strength" as the Shapley value over model-pool coalitions, with leave-one-out contribution as a tractable proxy (n+1 runs).
- Across 5 pools × 12 methods × 12 datasets, individual and compositional strength are largely uncorrelated (12 of 15 pool/method-family settings), weakest for general-purpose models.
- Compositional RL (C-RL) trains the policy inside a live multi-model system, rewarding the system outcome; the leave-one-out baseline cancels under group policy gradients, adding no rollout cost.
Results: a 27B policy with 3 larger frozen collaborators is the strongest collaborator in 21/24 settings, +4.0–5.4 pts avg (3× baselines), the only method never making the system worse; it transfers to unseen tasks/protocols and boosts pools containing GPT-5.5, Gemini, and Claude.
More from Research
- Laude Institute backs WaggleHaus, an open-source query engine for agentic queries over data lakes — lateinteraction · 2026-10-21
- UC Berkeley open-sources SkyDiscover, an AI framework that discovers algorithms — CShorten30 · 2026-10-21
- Another exoplanet candidate found — and a post-mortem on why it was missed — IgorCarron · 2026-10-21
- AI agents with $10,000 budgets still fail open-ended research: GPT-6 Astra and Fable 5.1 papers rejected — sayashk · 2026-10-21
- Stanford team launches ScholarCatalyst, a benchmark for literature retrieval that inspires research — lateinteraction · 2026-10-21
- Simple symmetric Venn diagrams with 17, 19 and 23 curves found via AI-assisted search, verified in Lean 4 — pickover · 2026-10-21