C-RL details: LOO baseline cancels out in GRPO, no extra rollouts needed
shangbinfeng · x · 2026-10-21
This thread post adds method details for Compositional RL (C-RL): the policy is trained inside a live multi-LLM system — frozen collaborators plus the collaboration protocol form the environment, and the system's answer is the reward.
Key trick: the leave-one-out baseline cancels out in GRPO's group-relative advantage, so training on Shapley marginals requires no extra rollouts and no access to collaborator weights.
More from Research
- Laude Institute backs WaggleHaus, an open-source query engine for agentic queries over data lakes — lateinteraction · 2026-10-21
- UC Berkeley open-sources SkyDiscover, an AI framework that discovers algorithms — CShorten30 · 2026-10-21
- Another exoplanet candidate found — and a post-mortem on why it was missed — IgorCarron · 2026-10-21
- AI agents with $10,000 budgets still fail open-ended research: GPT-6 Astra and Fable 5.1 papers rejected — sayashk · 2026-10-21
- Stanford team launches ScholarCatalyst, a benchmark for literature retrieval that inspires research — lateinteraction · 2026-10-21
- Simple symmetric Venn diagrams with 17, 19 and 23 curves found via AI-assisted search, verified in Lean 4 — pickover · 2026-10-21