LLM-generated reviews hit ICLR/NeurIPS papers; CMU essay warns of review monoculture
brwilder · x · 2026-11-07
CMU researcher Bryan Wilder shares that a paper from his group received reviews at NeurIPS (3/3) and ICLR (2/3) detected as 100% LLM-generated by pangram — "we've already incorporated all the Claude feedback we intend to."
His new essay, "Equilibrium effects of LLM reviewing," argues:
- The four roles for LLMs in peer review — author feedback, error detection, assisting human reviewers, and serving as reviewers — have very different consequences
- Reasoning must focus on the ecosystem-level equilibrium, not immediate output accuracy
- "LLM as reviewer" (as in AAAI's pilot) could centralize research evaluation, creating negative long-run incentives and monoculture in assessment
Short-term agreement between LLM and human reviews doesn't prove the practice is benign long-term.
Related event: CMU researcher finds 4 of 5 reviews for their paper were LLM-generated(3 posts)→
More from AGI Musings
- New framing: AI may commoditize work rather than automate jobs away — soumitrashukla9 · 2026-11-07
- Alignment thought experiment: treat utility functions as PII under a Kantian veil — jessi_cata · 2026-11-07
- Default to human-robot teams: why research should be judged on quality, not AI use — _akpiper · 2026-11-07
- Dozens of AI-discovered drugs in the clinic: rebutting Vertex chairman's claim that AI doesn't work — chaitjo · 2026-11-07
- 'We can befriend them, but they are not befriending us': NYRB revisits AI intimacy — _akpiper · 2026-11-07
- Gary Marcus mocks Anthropic: begged for regulation 6 months ago, now 'voters were actually listening?' — GaryMarcus · 2026-11-07