Should LLMs be first-pass reviewers for every scientific paper? Researchers say yes
anshulkundaje · x · 2026-09-21
- Computational biologist Anshul Kundaje amplifies Sam Sinai's proposal to formally add AI review to all scientific publishing.
- Sinai says he regularly runs Nature/Science-level papers through LLMs, and they correctly flag issues that human reviewers astonishingly missed.
- His proposal: each venue uses an AI reviewer with a prompt aligned to its standards to produce a first-pass review; human reviewers then read the paper plus the AI review, modify it independently or collaboratively, and take responsibility for the final verdict — even a simple "I take responsibility that this AI-assisted review is accurate."
- He criticizes the status quo as a "Werewolf game": an AI reviewer is clearly in the room, but everyone pretends it's all humans, with no transparency. Humans keep final say.
More from AGI Musings
- If superhuman AI researchers are imminent, will capability problems be easier than safety ones? — ronbodkin · 2026-11-12
- "AI is a terrible teacher": viral pushback against personalized AI tutoring hype — RachelVT42 · 2026-11-12
- OpenAI, Anthropic and DeepMind job posts signal PMs must write evals and code — aakashgupta · 2026-11-12
- Architecture study sparks monitoring concerns over a multi-hop planning spike — ChowdhuryNeil · 2026-11-12
- PowerShell creator Jeffrey Snover at Ignite: AI revolution is an extinction event for those who don't adapt — dfinke · 2026-11-12
- Worst AI use: signaling you don't care about things you should — Afinetheorem · 2026-11-12