Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG
Suting Chen, Peichun Hua, Yunming Xiao
cs.CL, cs.IR, cs.LG
2026-09-28
RINSE scores RAG evidence sufficiency before generation: 0.837 pairwise across six datasets, vs 0.746 best prior and 0.784 frontier API, at 36.5 ms on one GPU.
Retrieved passages in RAG can name the right entities and still omit the fact an answer needs. Semantic search always returns a top-k list. The system still has to choose among answering, searching again, and abstaining, and retrieval does not say whether the evidence covers the question.
Generators do not fill that gap. Twelve models (135M-32B), told to reply unanswerable when documents lack the answer, still answered 40.0-99.3% of 300 insufficient records and refused up to 29.7% of 300 sufficient ones. Phi-4 answered 40.0% of the insufficient set; SmolLM2-135M answered 99.3%. Median latency for a greedy answer of at most 24 tokens was 150-855 ms on the 7-32B models. The gap should be caught before that call.
Training the generator to abstain through fine-tuning, preference alignment, or RL tracks whether that model knows the answer. That lines up with evidence support only when parametric knowledge already matches the retrieved text, the assumption RAG is meant to drop. On NeoQA, whose news is fabricated so answers must come from the supplied passages, four methods that read a generated answer rank sufficient above insufficient at 0.503-0.520 (chance 0.5).
Sufficiency belongs to the question and the passage set, not to the model that will read them. The task is to decide, before any answer is decoded, whether the evidence contains what a complete answer needs, including intermediate facts.
Evaluation has to precede the scorer. Dropping the relevant passage or pairing evidence with an unrelated question leaks the label through word overlap, set size, or position. On SQuAD 2.0, the share of question words in the passage separates answerable from unrelated questions at 0.993, and falls to 0.573 against annotator-written unanswerable questions about the same passage. Three paired constructions hold selected cues down: substitution keeps the question and the passage count while swapping a required passage for a related distractor; deletion removes required support on NeoQA, where sets already range from 1 to 120 passages and count alone scores 0.563; question swap keeps the evidence and pairs it with a related question the passages cannot answer. The metric is within-group pairwise accuracy, the share of pairs where the sufficient side scores higher, ties counting as 0.5.
Evidence falls short in three ways: part of the question is uncovered; passages stay on topic but carry no answer span; facts sit across passages. RINSE assigns one non-decoding reader to each, subtracts a size trend, and sums with equal weights. The content reader splits the question into overlapping 4-subword windows and takes each window's best match from a single source, using IDF-weighted overlap and a 118M multilingual reranker. The span reader is a 125M RoBERTa extractive model trained on Natural Questions with explicit nulls; it returns the margin of best-span logit minus no-answer logit. The latent reader runs frozen Qwen2.5-1.5B-Instruct once over the question and the top five reranked passages, then projects the layer-17 final token onto a sufficient-minus-insufficient direction. Backbones stay frozen. The content head, latent direction, and normalization are fit with the evaluated family held out. Weights stay equal because the best reader changes by dataset. The output is a ranking score, thresholded at deploy time to answer, abstain, or retrieve more.
Pairwise accuracy on six datasets (chance 0.5):
| Method | Mean PA | Worst dataset |
| RINSE | 0.837 | 0.684 (SQuAD 2.0) |
| Self-RAG | 0.746 | 0.644 |
| Sufficient Context API | 0.784 | 0.676 |
| UPR | 0.721 | 0.541 |
| Local 1.5B Sufficient Context | 0.540 | 0.514 |
RINSE scores 0.684, 0.926, 0.773, 0.853, 0.813, and 0.973 on SQuAD 2.0, IIRC, NeoQA, HotpotQA, MuSiQue, and fresh Wikipedia. It leads Sufficient Context on GPT-6 Luna on five of six datasets, mean paired difference +0.053. Self-RAG wins SQuAD 2.0 at 0.752. On NeoQA the content reader alone reaches 0.799 and the span reader is near chance at 0.508; answer-reading methods all land at 0.503-0.520.
On 3,149 BM25-versus-e5-base-v2 pairs where one top-5 set contains every annotated support passage and the other does not, RINSE averages 0.813 against 0.674 for Self-RAG, 0.661 for UPR, and 0.800 for the frontier API. Both sides have five passages. UPR leads on SQuAD 2.0 (0.871 vs 0.813).
Ablations: 0.817 without latent, 0.799 without span, 0.779 without content. The best single reader's worst dataset is 0.583, against 0.684 for the sum.
Median latency is 36.5 ms per record on one A100 (17.4 records/s). The 603 ms 95th percentile is mostly the 118M reranker (465 ms); the 1.5B pass takes 29.0 ms. Gating the twelve generators cuts answering on insufficient evidence from 0.636 to 0.511 and saves 13.3% of generation time; answering on sufficient records falls from 0.861 to 0.783.
RAG stacks and multi-step search can run a local coverage check before calling a 7-32B generator, then abstain or retrieve again when passages are on topic but incomplete. The score is not bound to one model's weights, so it can sit in front of a closed API. A shared retrieval layer also needs this signal to decide when to fetch new pages.
The gain is a careful benchmark plus a cheap scorer, not a new generation recipe. The gate still withholds answers on sufficient records, and the time save is 13.3%. On SQuAD 2.0 near-misses, Self-RAG remains higher. The practical slot is a pre-generation filter, and a generator-independent reward for abstention training.
There is no standalone Limitations section; the constraints sit in the experiments.
Each cue check holds one shortcut at a time. After adding 20 passages to both sides of HotpotQA, minimum IDF coverage falls to 0.510, but an evidence-only classifier still scores 0.743. Of 832 NeoQA pairs with equal support counts, 831 still give the sufficient side fewer passages. RINSE leads local methods on those subsets; leftover surface cues are not wiped in one pass.
The span reader trains on Natural Questions; residual sentence overlap remains highest on HotpotQA and MuSiQue. Adding 5-20 passages to both sides drops mean PA from 0.847 at k=0 to 0.804 at k=20. Fresh Wikipedia questions are synthetic and may inherit generator bias. The gate uses 600 records and cuts recall on sufficient items. Equal weights let the span reader drag NeoQA below content-only. Training sets hold at most 22 passages while NeoQA reaches 120, so size correction extrapolates. The headline metric is ranking PA, not end-to-end QA accuracy. Cases where evidence is sufficient but the generator still fails, or insufficient but parametric knowledge happens to be right, are not measured directly.