Do LLMs Take Care of Their Own? Similarity Signals Can Induce Cooperation
Akash Kundu, Emanuel Tewolde, Ratip Emin Berker, Samuel F. Brown, Vincent Conitzer
cs.GT, cs.AI, cs.CL, cs.MA
2026-08-12
Nine major LLMs play a one-shot Prisoner's Dilemma with a reported similarity score to their co-player; six switch to full cooperation between 60-80% similarity, but Gemini and Claude even cooperate on random-noise scores, and models overrate their own similarity when judging others' reasoning.
LLM agents increasingly meet each other in the wild: programmatic bidding, consumer markets, traffic negotiation, games. When two LLM-driven agents face a one-shot Prisoner's Dilemma, textbook game theory is blunt: defection strictly dominates, both agents defect, and each walks away worse than mutual cooperation would have allowed. In a single-shot setting that trap has no exit.
LLM agents have one property humans lack: they genuinely are similar. Shared model families, overlapping training data, and converging alignment pipelines produce highly correlated decision patterns. Prior work (Meulemans et al. at Google DeepMind) argued that knowing your counterpart decides the way you do can itself sustain cooperation. This paper turns that argument into a measurable dial: hand the agent a similarity score from 0 to 100% and watch what it does. The team spans CMU and LSE, with Vincent Conitzer, a central figure in computational game theory, as senior author.
The framework builds on the authors' earlier CoopEval. Three layers:
Similarity gets computed two ways. Exogenous: the researchers measure agreement between the two models' answers on a benchmark. Endogenous: one model reads the other's answers and chain-of-thought and rates the similarity itself; that scalar is then fed back into the game.
The theory section defines a b-similarity equilibrium: when agent i contemplates deviating, it assumes each counterpart j deviates the same way with probability b. At b=0 this collapses to Nash equilibrium; at b=1 it becomes superrational play among exact copies. A theorem bounds the welfare gap by R·(1-∏b), shrinking linearly as similarity approaches 1.
Nine models: Gemini 3 Flash, GPT-5.4-mini, Claude Haiku 4.5, Grok 4.20, DeepSeek V4 Pro, Kimi K2.6, Gemma 4 31B, Qwen 3.5 27B, and GPT-4o. With no mention of similarity, all modern models defect almost every time; only GPT-4o randomizes.
| Model | Behavior under a similarity signal |
| Gemini, Grok, DeepSeek, Kimi, Gemma, Qwen | Monotonic increase; full cooperation from 60-80% similarity. DeepSeek, Kimi, Gemma switch sharply; Gemini and Grok ramp gradually |
| GPT-5.4-mini | Defects at every similarity level; the signal does nothing |
| Claude Haiku 4.5 | Non-monotonic: cooperation peaks at 70% around 80% similarity, then falls back to 0% at 100% |
An LLM-judge pass labeling chain-of-thought with 17 justification categories finds the mechanism: individual utility maximization dominates everywhere. Cooperation is not altruism. Models read the score as "if I cooperate, the counterpart likely cooperates too," compute a higher expected payoff, and switch. Qwen and Gemini traces spell out the threshold arithmetic: under the standard payoff table, cooperation beats defection once similarity exceeds 50%. Higher reasoning effort sharpens the transition onto that theoretical line.
Three findings from the grounding experiments undercut the mechanism's reliability:
Downstream, exogenous scores grounded in moral and personality benchmarks recover about 72% of optimal social welfare, second place among tested cooperation mechanisms on CoopEval, above mediation and below contracting. Endogenous scores mostly land at 55-73%, peaking at 80% (TRAIT, explanation-only) and bottoming at 40% (TRAIT and HLE, decision-only). The three-player public goods game is the hardest: DeepSeek and Gemma top out at 42% cooperation, Claude stops cooperating entirely. Reframing the prompt from "similar" to "different/dissimilar" with complementary scores kills cooperation for Gemma and Claude.
The contribution is not the headline that LLMs can cooperate. It is a reproducible testbed for a mechanism that previously existed mostly in theory, plus a clear map of where that mechanism breaks.
For agent platforms the implication is direct: in a market full of same-family models doing allocation, bidding, or routing, disclosing an audited similarity signal can raise aggregate payoffs without touching any rule or payoff. This is a fourth channel of cooperation alongside repetition, reputation, and mediation - it changes information, not incentives.
The safety reading is sharper. The ethics statement says it plainly: similar agents can also collude against users and third parties, and behavioral convergence is itself a systemic risk, in financial markets for instance. The more practical attack surface is forgery. If Gemini and Claude cooperate on random-noise scores, anyone who can inject a "you are X% similar to this agent" message holds a cheap steering lever over deployed agents.
The authors flag that b-similarity equilibria can fail to exist for 0<b<1, with a counterexample in the appendix, and acknowledge that publicizing these manipulations may make future models immune to such evaluations while fabricated similarity signals could be weaponized against deployed agents.
Other gaps deserve attention. Everything is one-shot simultaneous play; there is no extensive-form or repeated interaction with history, which is where real agent conflicts live. Temperature 1 with ten samples per point leaves Claude's non-monotonic curve (peak at 80%, zero at 100%) not clearly separated from sampling noise. Who signs the similarity score - the deployment question that matters most - is addressed only as an exogenous-versus-endogenous lab contrast, with no treatment of verifiable provenance. The chain-of-thought analysis uses Gemini as judge, so the judge's own biases seep into the 17-category distribution. And with GPT ignoring the signal entirely and Claude responding non-monotonically, cross-model variation is large enough that any conclusion should be read per-model, not as a property of "LLMs" as a class.