When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation
Sy-Tuyen Ho, Minghui Liu, Furong Huang
cs.LG
2026-09-18
One recursive mix of synthetic ICLR reviews compresses ratings and cuts same-paper diversity about 11%. TrustReviewer hits 75.40% exact match with curated SFT and steering.
LLMs already draft reviews, edit reviews, and assign scores. Once that text is public, it enters later training corpora, and the next reviewer model learns from the previous model's judgments. That is recursive training applied to peer review. Work on model collapse says distributional tails can vanish under generated data. Whether scientific judgment itself narrows still needed a controlled test.
The Maryland group isolates one recursive step and one base model, and asks a sharper question: after synthetic reviews enter training, do scores get easier, harsher, or merely more alike.
Start from Llama 3.1 8B Instruct. Fine-tune with LoRA on official ICLR reviews from 2018–2023 to get M1, then let M1 write synthetic reviews for ICLR 2024 papers. Successor models M2 continue from M1 on mixtures that always use three reviews per paper, with 0%, 33%, 66%, or 100% synthetic (0/1/2/3 generated reviews). Optimization settings are shared. The paper says "official" rather than "human", because reviews after ChatGPT may already contain unobserved AI help.
A held-out set of 2,000 papers across years is reviewed three times per model. Metrics: rating mean, standard deviation, entropy, same-paper pairwise semantic distance, and corpus spread around the centroid.
TrustReviewer drops the recursive second stage. Official reviews from 2018–2025 are filtered into 112,743 examples, about 1.9 billion tokens, and the base model is SFT'd in one pass. At inference, 5,000 official-versus-model pairs estimate a last-layer steering vector with strength 0.15, added only at the current token. No extra training, no extra labels.
Official ratings have standard deviation 1.73 and entropy 2.38. M2 at 0% synthetic already sits at 1.63 and 2.31. Thirty-three percent synthetic compresses those to 1.44 and 2.14; 66% and 100% stay below the 0% baseline. Means are not monotonic: 5.30 at 0%, 5.85 at 33%, 5.70 at 100%. Diversity shrinks. Leniency does not march in one direction.
Same-paper semantic distance falls from 0.159 to 0.142 (about 11%). Corpus spread falls from 0.609 to 0.579 (about 5%). Both drop monotonically with synthetic exposure.
| Model | Exact match | MAD | Rating entropy |
| Llama 3.1 8B Instruct | 33.93% | 2.679 | 1.53 |
| Qwen3.6-35B-A3B | 61.85% | 1.335 | 1.94 |
| OpenReviewer | 73.10% | 1.113 | 2.10 |
| TrustReviewer, no steering | 73.85% | 1.077 | 2.13 |
| TrustReviewer | 75.40% | 1.079 | 2.18 |
| Official reviews | — | — | 2.38 |
Steering adds 1.55 points of exact match and lifts entropy from 2.13 to 2.18; MAD barely moves. Same-paper distance dips slightly, remaining above Llama and Qwen and below OpenReviewer. OpenReviewer's corpus is private; this one is released.
If conferences keep feeding model-written reviews into the next reviewer, a single recursive step is enough to squeeze the rating distribution and pull same-paper opinions together. The failure mode is a smaller judgment space, not a uniform shift toward harsh or kind. The mitigation is concrete: filter short, repetitive, malformed supervision, then nudge activations with the official-versus-model difference. Anyone SFT'ing a reviewer on unfiltered synthetic reviews is training the collapse on purpose.
Exact match tracks official score distributions, not scientific truth. High diversity is not high quality.
The appendix is explicit. The experiment is one recursive step, one 8B family, one ICLR domain. Whether collapse compounds over generations, or transfers across scales and fields, is open. Diversity is not quality; a homogeneous set can still be right. Exact match and MAD measure agreement with official ICLR ratings, which contain errors, disagreement, and possible AI. Embedding distances are a proxy from one embedder, not proof that reviews raise different valid scientific issues. There is no human grading of critique correctness, specificity, or coverage. Ending M1's official data in 2023 lines up with Llama 3.1's December 2023 cutoff and reduces the chance that 2024 reviews were in pretraining; it does not eliminate it.