From Solo to Social Learning: Characterizing Recursive Social Improvement in LLMs
Kunal Jha, Max Kleiman-Weiner, Natasha Jaques
cs.MA, cs.AI, cs.LG
2026-09-30
LLM agent populations that copy peer skill files learn more cheaply, but at matched token cost they never beat independent learners, and exchange leaves fewer discoveries.
Self-improvement methods for LLM agents (prompt optimization, reflection, evolutionary search) optimize one system at a time, while multi-agent frameworks put many models on a shared goal. A University of Washington team asks the question in between: if a population of self-improving agents, each rewarded only on its own tasks, can see the skills its peers produce, does the population beat independent learners spending the same tokens? They call this recursive social improvement, and the stakes are practical. Agents already download each other's skill libraries from GitHub, and OpenAI coordinated roughly 10,000 agents on its Navier-Stokes effort. Behavioral ecology has long warned about the tension (Rogers, 1988): copying pays only if someone keeps exploring independently, and if everyone waits to copy, the supply of discoveries dries up.
The core design puts learning and acting on one token budget. Each round, an agent first picks a learning action: revise one of its own skills, observe a peer (acquiring that peer's latest skill and score), or skip. It then selects any skill from its library and executes it on a task. Everything draws on the same 4,096 completion tokens per round, so an agent that reasons too long about whom to copy runs out of budget before acting.
The key measure is the social advantage at matched cost, ΔJ(C): the population with peer observation against the same models with observation disabled, compared at the same cumulative spend. The authors separate efficient learning (more performance per token) from effective learning (beating independent learners at equal cost), because stopping search early also makes per-token returns look good.
Two stages: a controlled environment turning the Rendell et al. (2010) social-learning tournament into a 25-armed bandit played by 10 agents for 100 rounds, testing Qwen3-14B, Ministral-3-14B, and GPT-OSS-20B against UCB baselines; and skill evolution on IFBench, an instruction-following benchmark, where agents revise skill files with OpenEvolve (100 training examples, 200 held-out), with GPT-OSS-120B and GLM-5.3-Flash each running six five-agent populations.
The bandit does reward social learning for a well-designed policy: hierarchical UCB, which first applies UCB to choosing whom to learn from, earns 25.68 ± 2.87 more cumulative reward than solo UCB (p < 0.001). None of the three LLMs gets that benefit.
| Setting | Metric | Result |
| Bandit, matched cost | ΔJ(C), social minus solo | Qwen +0.14 ± 0.33; Ministral +0.37 ± 0.34; GPT-OSS −1.24 ± 0.45; none significant |
| Bandit | reward per token | Qwen −29.6%; Ministral −41.8%; GPT-OSS −51.9% |
| Skill evolution, matched tokens | held-out accuracy gap | GPT-OSS +0.14 ± 0.35 pp; GLM +0.98 ± 0.85 pp; neither significant |
The failure is not bad copying: a copied arm beats the best arm the copier had found itself in 93.2% of comparable GPT-OSS pulls. Two mechanisms hurt instead. Social populations collapse onto few arms once peers' good arms are visible, and reasoning about whom to observe burns the action budget: Qwen's social agents complete a pull in only 81.0% of rounds versus 98.1% solo. Two targeted interventions each fix their target. Reserving 2,048 tokens for execution raises Qwen's completion by 19.53 percentage points (p < 0.001), and forcing independent search on 15% of rounds adds 1.27 distinct arms for GPT-OSS (p < 0.01). Neither closes the gap to solo.
In skill evolution the two models diverge. GLM observes rarely and early (8.4% of rounds) and holds 1.31 ± 0.61 points higher held-out accuracy during learning, but solo catches up by the end. GPT-OSS observes often (35.9%), gains no accuracy, and spends 17.34 ± 2.08 million fewer learning tokens. Recursive transmission genuinely happens: at the final round, 83.3% of GPT-OSS and 56.7% of GLM deployed skills descend from a copy-then-revise chain, yet GPT-OSS social populations trace their five deployed skills back to a single founding lineage on average, versus 4.83 for solo. Handing the when-to-copy decision to a simple rule, UCB or even uniform random, cuts learning tokens 76-78% versus full solo OpenEvolve with final accuracy within 1.03 points.
For agent infrastructure work this is a negative result worth remembering: leave multiple self-improving agents side by side and they do not automatically benefit from each other, as long as the LLM itself decides when and whom to copy. Copying does help individuals and cuts population compute, and a simple UCB rule, or even random choice, captures most of that saving. The bottleneck is the learning decision itself, which looks like a trainable capability rather than a freebie from building the sharing mechanism.
The authors list: three models in the bandit and two models on a single benchmark; peers always listed in the same order, so peer preference cannot be separated from position effects; comparisons made at tokens actually spent rather than a pre-fixed budget; agents keep only their latest observation in full; one model per population; fixed decision prompts. Two further discounts from reading closely: part of GPT-OSS's token saving is mechanical, since observing a skill avoids the 100-example evaluation every revision requires, which the authors acknowledge; and GLM's mid-learning advantage vanishes by the endpoint, making it acceleration rather than a final-performance win.