Ai2 Mined ICLR 2026 Reviews to Build Its Novelty Benchmark: 154 Strong vs 145 Weak Papers
allen_ai · x · 2026-10-28
Ai2 built its benchmark by mining ICLR 2026 peer reviews for explicit originality comments, keeping papers where reviewers agreed on (non-)originality and whose scores were unusually high or low within their area: 154 papers with stronger evidence of novelty (e.g., "a completely new, critical, and timely research problem", top 10% in area) versus 145 with weaker evidence (e.g., "the theoretical contribution is not highly novel", bottom 10%).
More from Research
- NVIDIA open-sources Flora, a model suite that auto-labels robot videos for training GR00T — qinzytech · 2026-10-28
- Multimodal GraphRAG models missing and contradictory evidence for storm damage assessment — JeremyCMorgan · 2026-10-28
- SIMIT: VLMs self-improve at test time by imagining practice problems, +7.2% on 17 benchmarks — delliott · 2026-10-28
- Mistral sponsors Open Math OS with zero data retention, powering a Kimi K3/DeepSeek math platform — GuillaumeLample · 2026-10-28
- CROP wins EMNLP 2026 Outstanding Paper: conformal certification for reasoning trace prefixes — hanjie_chen · 2026-10-28
- OSU NLP Group Wins Three Paper Awards Same Day at EMNLP 2026 and NeurIPS Workshop — hhsun1 · 2026-10-28