ClaimNov paper: claim-level novelty checks cut false alarms from 11.8% to 1.5%
rohanpaul_ai · x · 2026-11-01
The full UIUC paper "Do AI Scientists Know What's New?" (arXiv:2610.36511) builds a temporal-ground-truth framework across four datasets of 1,060 proposals and audits existing novelty checks: high rejection rates on non-novel work correlate with high false alarms on ideas later realized by human researchers, and whole-proposal checks can only flag prior art by rejecting everything.
- ClaimNov catches 97% of disguised papers at a 1.5% false-alarm rate, cutting the LLM judge's false alarms from 11.8% to 1.5%
- It correctly rejects 367/380 disguised proposals and names the source for 366
- It rejects only 3% of ICLR-accepted proposals with reviewer-disputed contributions and keeps 91/100 novel proposals with one spliced prior contribution (vs 76 for the strongest existing check)
- Code and data are open-sourced at github.com/emmatliu/claimnov.
Related event: UIUC Paper Finds AI Novelty Checks Reject 96% of Real Research Ideas(2 posts)→
More from AGI Musings
- Debate: If local models get "good enough", OpenAI and Anthropic are toast — TheZachMueller · 2026-11-01
- We are already governed by AI to some extent, says Jarovsky — LuizaJarovsky · 2026-11-01
- Cyberpunk creator Mike Pondsmith: an ASI could serve humanity with 10% of its compute — Akumetsu_971 · 2026-11-01
- Ramayana's 6-Year Handcrafted Art Pushback Ignites AI Slop Debate — vishalmisra · 2026-11-01
- At least the cyber apocalypse is arriving before humanoid robots scale — mimi10v3 · 2026-11-01
- The case for everyone running their own local LLM, from a liberal perspective — SoloGen · 2026-11-01