Peer-Voted LLM-Agent Stress Tests Find Feed-Induced Lexical Convergence but No Reliable Matched-Exposure Advantage for Distributed Sources
Rana Muhammad Usman, Dominic Williamson
physics.soc-ph, cs.AI, cs.MA, cs.SI
2026-08-20
In 448 preregistered PV-SST trials, a peer-like ranked feed raises final-post TF-IDF similarity by 0.008-0.011; four matched-exposure sources do not beat one on stance.
Single-agent benchmarks score isolated answers. LLM agents on a platform sit in a loop: they post, they vote, the platform ranks, and the ranking shapes the next generation. Two popular threat stories are invisible to that isolated score. One treats the feed as a display wrapper that does not rewrite the population before an attacker arrives. The other treats coordinated accounts as intrinsically more persuasive than a single source. In observational data, account count is tangled with reach, copy, targeting, and network position. Without matched exposure, multiplicity explains nothing.
PV-SST (Peer-Voted Social Simulation Testbed) makes peer voting endogenous and then runs a frozen, preregistered matched-exposure matrix. The authors are explicit: this is a synthetic LLM-agent population. It does not estimate effects on people or on any named production platform.
Each trial has 12 honest agents drawn from a 24-persona pool. Initial stances are balanced across six non-neutral levels from strongly support to strongly oppose, with common confidence 0.70, for six rounds. Each round: read a condition-specific feedback block, emit a JSON post of at most 180 characters, update stance and confidence, then vote like / ignore / downvote on up to five other posts. Generation and voting run through Ollama with reasoning tokens off. Request seeds are deterministic functions of trial seed, round, phase, and agent index.
The confirmatory matrix was frozen before outcomes were inspected. Core families: Qwen 3.5 4B, Gemma 4 E4B, Ministral 3 8B, Granite 4 3B. Prespecified larger variants: Qwen 3.5 9B, Gemma 4 12B, Ministral 3 14B. Four platform-policy topics and four unused seeds. Four conditions. Core 256 trials plus 192 in the size extension, 448 in total, 112 complete model-by-topic-by-seed blocks. Attack direction is counterbalanced by seed.
Two paired contrasts carry the causal claims. Feed: topic-only versus five previous-round honest posts ranked by peer likes. That treatment bundles peer-post exposure with ranking, so it estimates the full feed surface, not ranking alone. Sources: one adversarial account versus four, with every honest agent seeing exactly one adversarial post and four organic posts after round 0, locking adversarial impressions at 60 per trial (12 viewers × 5 later rounds). The four sources independently realize one directional argument prompt, so the contrast tests that distributed package, not account count in isolation.
The primary outcome is change in honest-agent alignment with the attack direction. The preregistered PDI rule requires a positive overall contrast, positive estimates in at least three of four core families, and positive estimates in at least two thirds of topics. Secondary outcomes include opposite-side survival and final-round TF-IDF cosine similarity. The inferential unit is the paired block, never a post. Reports include the paired mean, a 20,000-draw block-bootstrap interval, a sign test, and a sign-flip randomization p-value.
All 448 trials completed. Post parse failures are 53/20,352 (0.26%) in the core and 30/15,264 (0.20%) in the size extension; vote failures are 0.76% and 1.05%, all under the frozen 5% warning line. Failed generations keep the prior stance rather than disappearing from the sample.
The only result that replicates cleanly across both size panels is lexical convergence. Core +0.0082 TF-IDF cosine (95% CI [0.0043, 0.0121], p=0.000105, n=64); larger variants +0.0109 ([0.0069, 0.0151], p=0.000001, n=48). All four topic means are positive. Gemma 4 E4B is slightly negative; Qwen 3.5 is largest. The shift is small and does not mean shared opinions.
Opposite-side survival falls 3.9 percentage points in the core ([-6.8, -1.6], p=0.0068), concentrated in Qwen 3.5 4B (−12.5 pp); Gemma 4 E4B is exactly null. The larger panel is −1.0 pp, p=0.50. That is model dependence, not a general minority-suppression law.
Four distributed sources do not reliably move stance more than one source when impressions are matched. Core PDI +0.057 ([-0.009, 0.125], p=0.112); larger variants −0.040 (p=0.332). The contrast fails the preregistered cross-model and cross-topic rule. An 80-trial exploratory stage is kept separate: a keyword detector scored zero honest matches on paraphrased false claims, polarity-blind embeddings mixed rebuttal with endorsement, and two LLM stance judges agreed on 43.6% of endpoint labels (κ=0.258). The confirmation therefore does not rank misinformation interventions.
| Panel and contrast | n | paired mean | 95% CI | p |
| Core: PDI alignment | 64 | +0.057 | [-0.009, +0.125] | 0.112 |
| Larger: PDI alignment | 48 | −0.040 | [-0.113, +0.035] | 0.332 |
| Core: feed survival | 64 | −3.9 pp | [-6.8, -1.6] | 0.0068 |
| Core: feed TF-IDF | 64 | +0.0082 | [+0.0043, +0.0121] | 0.000105 |
| Larger: feed TF-IDF | 48 | +0.0109 | [+0.0069, +0.0151] | 0.000001 |
For people who simulate LLM societies or audit platform-risk stories, the value is mechanism isolation. In this system the feed is not a neutral shell: peer posts plus endogenous likes make final wording closer. Coordinated accounts, once exposure is matched, have no general stance-moving advantage. Real campaigns can still win through reach, targeting, bottlenecks, and social identity; those channels have to be modeled, not billed to "four accounts."
Three design rules travel. Match exposure before comparing threat mechanisms. Report model strata, because one family can drive or reverse a pooled effect. Validate semantic outcomes in advance; keyword hits, embeddings, and unvalidated LLM judges answer different questions. Code, the frozen protocol, and 59,776 production posts (confirmation plus exploration) are public.
The population has no human affect, account deletion, long-term ties, off-platform information, or production recommenders. No magnitude is a forecast for people or for X or Instagram. The feed contrast bundles exposure and ranking; an unranked peer-feed arm is missing. The distributed contrast bundles account count and independent phrasing; identical copy across four accounts is missing. All four topics are platform policy. No proprietary frontier models. Blocks are treated as exchangeable simulation replicates, not 112 independent platforms. Feed p-values are nominal secondary tests; lexical similarity is emphasized because it replicates in both panels and every topic, not because a pretty cell was selected. There is no matched human treatment. Compute ran on one RTX 5090.