510k peer reviews show LLM-era homogenization: ICLR critique overlap jumps to 21.3%
xuanalogue · x · 2026-11-21
Analyzing 510k conference and journal reviews, Ilya Gusev finds critique overlap between reviewers has risen sharply since ChatGPT: same-stage ICLR review overlap jumped from 12.0% in 2026 to 21.3% in 2027, with year-on-year rises at NeurIPS, ICML, and the F1000 and PLOS journals.
Running the MELD LLM-text detector on all reviews, the paper shows two MELD-flagged reviews of the same paper overlap more than unflagged ones: by 3.0 points at ICLR 2024–2025, 4.5 points in 2026–2027, and 5.4 points across five journal sources. A third ICLR review added 5.46 new problems beyond the other two in 2027, down from 6.12 in 2026.
Takeaway: LLM-written reviews are less distinct from each other, eroding the diversity of critiques that multi-reviewer peer review is designed to provide.
Related event: 510K Reviews Show ChatGPT Is Homogenizing Peer Review(2 posts)→
More from Research
- Colgate says LLM role-play can generate ~90% accurate survey data, challenging market research — ZeroStateReflex · 2026-11-22
- Continual Causality Bridge returns at AAAI-27, novel-work deadline Dec 1 — mundt_martin · 2026-11-22
- Harvard lab uses Biomni AI agents to decode the hidden "grammar" of DNA — anshulkundaje · 2026-11-22
- PredActor Cuts Diffusion to 5% of Forward Pass for Low-Latency Humanoid Control — chris_j_paxton · 2026-11-22
- Claude's System Card Reveals Protein-Sequence Training, Sparking Debate on Bio Foundation Models — chaitjo · 2026-11-22
- Axiom offers to pay biotechs to run drug programs while they keep 100% of assets — cjmaddison · 2026-11-22