NeurIPS Peer Review is Highly Random: 50% of Spotlights Were Rejected in RCT
NeelNanda5 · x · 2026-07-24
AI researcher Neel Nanda shared practical advice for researchers going through the NeurIPS rebuttal phase. He advised authors not to take the review results too personally or be overly defensive, emphasizing the inherent randomness (RNG) in academic peer review.
Citing data from a randomized controlled trial (RCT) conducted before the use of LLMs, he pointed out that 50% of NeurIPS Spotlight papers were rejected by an independent review committee. This highlights that review outcomes involve significant luck, and researchers should focus on engaging constructively with genuine feedback.
More from Research
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- Ben Recht: The world needs a truly open language model, and it's easier to build than you think — beenwrekt · 2026-10-26
- New Blog Breaks Black-Box Distillation Into Five Practical Steps — chrisrohlf · 2026-10-26
- DetectorScope turns black-box AI text detectors into auditable ones with sparse autoencoders — _akhaliq · 2026-10-26
- OPST paper shows LLMs self-improve with zero human data, guided only by topic descriptions — _akhaliq · 2026-10-26
- Tencent's HiSTEVE 1.0: hierarchical VLA agent plays Minecraft with native keyboard and mouse, hits 85% SOTA — _akhaliq · 2026-10-26