arXiv paper: Jev-as-a-Judge — non-generative verifier matches frontier LLM on RAG claims at 1/187 cost
rohanpaul_ai · x · 2026-10-27
The arXiv paper (2610.20634) behind the thread, by Alden Do Rosario and Kirollos Atef of CustomGPT.ai.
A non-generative verifier (Jev) statistically ties a frontier LLM on RAG claim verification (-0.56 points) at 187x lower cost, but passes 23.2% of unsupported claims vs the frontier model's 14.1%. Confidence-gated cascading reaches 74.10 at $2.92 per 1k claims — yet matched random escalation does as well, and the cascade costs 61x more than pure Jev for a statistically insignificant 0.83 points. Equal accuracy conceals unequal errors. Predictions, prompts, pricing and code are released.
More from Research
- DREAM symposium at NeurIPS 2026: agentic AI for disease biology, abstracts due Nov 20 — Pseudomanifold · 2026-10-27
- AI designs lipid nanoparticles to deliver RNA drugs into solid tumors — paraschopra · 2026-10-27
- Building an open benchmark to compare AI approaches to analog chip design — paraschopra · 2026-10-27
- Researcher building benchmarks to tell whether LLMs compute or recall answers — paraschopra · 2026-10-27
- AI-written Community Note debunks brain injury lens study: only 14 of 41 participants analyzed — NathanpmYoung · 2026-10-27
- Anthropic's Claude finds 2^o(n) 3-SAT algorithm, refuting the Exponential Time Hypothesis — rickasaurus · 2026-10-27