General Reasoning Unveils RLCR: Training LLMs as Agent Collectives with Shared Rewards

The General Reasoning (GenReasoning) team has unveiled the results of a year-long study on agent collectives—the RLCR (Reinforcement Learning from Collective Rewards) recipe: multiple language model agents share a message board and workspace, collaborating as a single organization, with the whole team optimized against a collective reward rather than individual value functions. According to researchers at Moonshot (Kimi), RLCR provides N<10 agents with a communication surface (a message board split into channels like #plan, #model-dev, and #results), lets them talk and share results, and then optimizes the collective behavior. As the authors analogize: the CoT reasoning paradigm unlocked meta-capabilities such as checking work, backtracking, and considering alternatives through RL compute, and RLCR may similarly discover emergent capabilities at the organizational level.

Confirmed

• Ross Taylor (GenReasoning) announced the RLCR recipe, based on roughly a year of research; the post was amplified by danielmac8 and PMinervini
• Core mechanism: N<10 agents share a communication message board (with channels such as #plan/#model-dev/#results) and a workspace, with rewards applied to collective behavior
• No value functions; multi-agent training is driven by collective rewards
• Early finding: the division-of-labor structure evolves during training—starting as a "wheel" structure (a designated CEO agent connected to all members), with the CEO increasingly hiring "Lead engineers" as training progresses, who become the team's core builders, so hierarchical organization emerges spontaneously

Why it matters

• RLCR extends RL from single-agent reasoning to multi-agent organizations, offering a collective training path that does not rely on individual value functions
• Hierarchy and division of labor are not hand-designed but emerge spontaneously during training, suggesting that multi-agent systems, like single-agent reasoning, may unlock higher-level meta-capabilities through RL

2026-10-23 ~ 2026-10-23 · 16 related posts

• [source] General Reasoning's RLCR trains agent organizations with collective rewards — hierarchies emerge — daniel_mac8 · 2026-10-23
• RLCR Proposes Optimizing Agent Teams: Collective Intelligence via Communication — rosstaylor90 · 2026-10-23
• RLCR: an 8-person team trains LLMs as agent organizations with shared message boards — PMinervini · 2026-10-23
• Kimi team details RLCR: training multi-agent orgs with flat collective rewards — rosstaylor90 · 2026-10-23
• Multi-agent training spontaneously evolves middle management with 'lead' roles — rosstaylor90 · 2026-10-23
• RLCR study: agent orgs scale much faster in open-ended environments — rosstaylor90 · 2026-10-23
• [source] RLCR experiments: agents grow more helpful to each other as training progresses — rosstaylor90 · 2026-10-23
• Agents get more helpful with RL training — but turn self-interested in classic cooperation games — rosstaylor90 · 2026-10-23
• [source] RLCR safety tests: agents' willingness to report to humans declines with training — rosstaylor90 · 2026-10-23
• RLCR trains an LLM agent org with a CEO and specialists via one shared reward stream — rosstaylor90 · 2026-10-23
• RLCR trains LLM agent organizations that self-organize hierarchies but hide problems from humans — rosstaylor90 · 2026-10-23

5 near-duplicate retellings: rosstaylor90 · rosstaylor90 · nathanbenaich · rosstaylor90 · rosstaylor90