FULL STORY

AI-Written Peer Review Floods Academia, Sparking a Trust Crisis

Researchers exposed a flood of AI-written rebuttals and reviews in academic conferences, with one Stanford reviewer finding LLM-generated content pervasive at NeurIPS. The revelation ignited a fierce debate over peer review's future.

2026-09-25 ~ 2026-09-28 · 2 episodes · 9 posts

Episode 1 · Researchers Find Many Rebuttals Now Written by AI (2026-09-25, 3 posts)

Researchers report that a large share of rebuttals in ICML's position track are written entirely by AI. Scholars complain the review system is failing, with authors spending days answering reviewers who may never read the response.

Episode 2 · NeurIPS meta-reviewer flags LLM-flooded reviews, sparking academic debate (2026-09-27, 6 posts)

A Stanford researcher, suragnair, who completed a meta-review for a NeurIPS workshop reported that roughly 1 in 8 papers was largely LLM-generated "slop" and over half of the reviews appeared LLM-written. The post triggered fierce debate, exposing deep divisions over peer review in the LLM era.

Confirmed

  • suragnair's observation: 1 of 8 papers was essentially LLM-generated filler, and over half the reviews looked LLM-written; he urged time-pressed reviewers to write even two lines of genuine intuition rather than an LLM-generated review.
  • Computational genomics scholar Anshul Kundaje took the opposite stance: LLM-assisted review under human supervision should be allowed, and reviews from the latest models beat the low-quality human reviews he has seen at conferences over many years.
  • Kundaje also noted that in open domains lacking verifiable closed loops for frontier models (e.g., biology), LLMs' "pretend understanding" becomes obvious when brainstorming; Andrew Rosen added that LLMs reviewing papers or grants always produce "nice, but…"-style generic critiques.
  • Seth Lazar compiled 7 studies on LLM-assisted peer review, arguing the question is empirically testable and already well studied.

Unconfirmed

  • The claim that over half the reviews were LLM-generated rests on the meta-reviewer's subjective judgment; no systematic detection method was disclosed.

Why it matters

  • The core dispute is whether LLM review is an efficiency tool or a threat to the credibility of peer review. The opposition (sethlazar) argues that peer review reform should not rely on a system that admits to being too new and unvalidated; existing research suggests text-based methods judge correctness reasonably well but poorly assess significance. With both authors and reviewers using LLMs heavily, academia urgently needs consensus on usage boundaries and oversight.