510k peer reviews show LLM-era homogenization: ICLR critique overlap jumps to 21.3%

xuanalogue · x · 2026-11-21

Analyzing 510k conference and journal reviews, Ilya Gusev finds critique overlap between reviewers has risen sharply since ChatGPT: same-stage ICLR review overlap jumped from 12.0% in 2026 to 21.3% in 2027, with year-on-year rises at NeurIPS, ICML, and the F1000 and PLOS journals.

Running the MELD LLM-text detector on all reviews, the paper shows two MELD-flagged reviews of the same paper overlap more than unflagged ones: by 3.0 points at ICLR 2024–2025, 4.5 points in 2026–2027, and 5.4 points across five journal sources. A third ICLR review added 5.46 new problems beyond the other two in 2027, down from 6.12 in 2026.

Takeaway: LLM-written reviews are less distinct from each other, eroding the diversity of critiques that multi-reviewer peer review is designed to provide.

Related event: 510K Reviews Show ChatGPT Is Homogenizing Peer Review(2 posts)→

Original post →

More from Research

Research channel →