Open-source RAG verifier catches broken chunks and citations LLM judges miss
hlcAlex · reddit · 2026-10-23
A Reddit user released rag-verifier, a free open-source tool targeting two unchecked gaps in RAG pipelines: whether chunks are self-contained and whether citations genuinely support claims. In one example, a contract Q&A answered "15% liquidated damages" citing only the consequent clause—the trigger condition (60+ days late) sat in the previous chunk, and the actual penalty was 0%, yet an LLM judge passed it. The tool combines a deterministic regex rule layer (catching dangling conditionals, connector starts; zero false positives across 20 real documents / 1388 chunks) with an LLM layer for citation faithfulness and chunk independence. It fails open rather than faking a pass. Demo and GitHub repo available.
More from coding & agent
- Dev Jokes: You Can Only Relax Using Claude on Stuff That Doesn't Matter — louisvarge · 2026-10-23
- PHP Has Been 'Dead' for 30 Years — and It's Still Not Dead — 0xkarasy · 2026-10-23
- VC: By 2027 companies won't be able to justify hiring non-agent engineers — latticecut · 2026-10-23
- Opus 5.5 codes an entire sci-fi movie title sequence—every frame and synthesized note — prasenx · 2026-10-23
- Local AI slide generation for privacy-constrained healthcare: HTML vs PPTX — AdRepulsive7837 · 2026-10-23
- MiniMax Design MCP goes live: create with Design inside Codex and other agents — Hailuo_AI · 2026-10-23