PremiseCheck Fact-Checks Peer Reviews at GPT-5.5-Level Accuracy for Half the Cost
Dingjie Song · hf · 2026-10-19
A Lehigh/UBC/Vector Institute paper tackles fact-checking of human and AI peer reviews. The team releases ReviewFact, a benchmark labeling factual errors in review comments across NLP, ML and natural sciences, measuring accuracy alongside cost and latency. Frontier LLMs check well but must read the full paper; cheap decision models fail on comments mixing several implicit assertions. PremiseCheck combines both: an LLM rewrites each comment into explicit premises without reading the paper, and a decision model checks each premise. It matches GPT-5.5 accuracy at about half the cost, and a quarter of the cost with less than half the latency on NLP reviews.
More from Research
- Team seeks Hilbert's 6th problem collaborators to build verifiable logic for AI x Science — CatAstro_Piyush · 2026-10-19
- Dream2Discern: Self-Generated Hard Examples Predict Multi-Agent Failures With 90%+ Accuracy — RutgersU · 2026-10-19
- BridgeWAM: latent bridge queries decouple video and action experts, 2.47x faster robot inference — Fudan-University · 2026-10-19
- NVIDIA's FC-SWE: Failure-Conditioned RL Boosts Multi-Attempt Resolution for SWE Agents — nvidia · 2026-10-19
- AVC-Reward: an event-centric framework for evaluating audio-video generation consistency — ZhejiangUniversity · 2026-10-19
- SciΠvot benchmark: frontier models prefer flawed research plans and lag human scientific judgment — Youran Sun · 2026-10-19