Peer review homogenizes in the LLM era: ICLR overlap jumps 12% to 21.3%
Ilya Gusev · hf · 2026-11-20
Ilya Gusev (Keenable/Amsterdam) quantifies homogenization of peer review in the LLM era.
Key findings:
- Claim-level critique overlap in same-stage ICLR reviews rose from 12.0% in 2026 to 21.3% in 2027, with year-on-year rises at NeurIPS, ICML, and the F1000 and PLOS journals
- A third ICLR review added only 5.46 new problems to the other two in 2027, down from 6.12 in 2026
- Using MELD, a detector of LLM-generated text, reviews identified as LLM-generated are less distinct from each other
The upshot: AI-assisted reviewing is converging critiques, weakening peer review's premise of independent readers finding different problems.
More from Safety
- SIP2Shell: one SIP text message to FreeSWITCH gives you RCE (CVE-2026-116779) — jedisct1 · 2026-11-20
- OpenAI co-hosts Hacktivate AI 2.0 policy hackathon in Brussels on Dec 1 — MartinSignoux · 2026-11-20
- Personal Agents Get You Wrong: Five Memory Failure Modes and How to Audit Them — sujingshen · 2026-11-20
- Tencent's detailed model safety card reveals work with startup Fangcun AI on misalignment evals — terryyuezhuo · 2026-11-20
- Zero-security dev's autonomous agent went from one-byte overflow to tcache poisoning in hours — Genaforvena · 2026-11-20
- TrustedSec: AI agents make attacker attention cheap, rewriting offensive security economics — cyb3rops · 2026-11-20