The Magnitude Mirage: Rethinking Confidence for Reasoning-Intensive Retrieval
Jamie Holdcroft, Abdelrahman Abdallah, Adam Jatowt
EMNLP 2026
cs.IR
2026-09-14
On 11 retrievers and 28 datasets, max similarity is near-random for logical and temporal RAG abstention; Score Gap and LSMV lift AUROC by up to 0.16 at zero cost.
Production RAG stacks often abstain by thresholding raw similarity. LangChain and LlamaIndex both encode the same assumption: a high score means retrieval worked.
Dense retrievers are trained for semantic proximity. Once a query adds a logical or temporal constraint, a document can sit close in embedding space and still violate the constraint that would make it useful. Two queries can share the same top score of 0.85 while their score lists look nothing alike: a hit isolates the right document; a miss is almost flat. The paper calls that false confidence the Magnitude Mirage.
The intervention is zero-cost Query Performance Prediction: take the top-k score list the retriever already produced, turn it into a confidence, and use that to abstain. Success is NDCG@k > 0, i.e. at least one relevant passage in the context window.
The baseline is MaxScore, which is just s1. Five alternatives read the distribution instead:
They slice 28 datasets into three cognitive tiers: semantic matching on four BEIR sets (SciFact, FiQA, NFCorpus, SciDocs), logical reasoning on 12 BRIGHT subsets, temporal reasoning on 12 TEMPO subsets. Eleven retrievers cover BM25, dense bi-encoders (BGE, E5, Contriever, SBERT, SFR, Qwen, Instructor-Large), and reasoning-tuned encoders (Diver, RaDeR, ReasonIR). Cutoffs are k in {5, 10, 25, 50}. Primary metric is abstention AUROC.
An end-to-end check feeds each retriever's top-5 BRIGHT documents to Qwen3.6-27B and lets GPT-4o judge answers against gold evidence, 19,376 query-retriever pairs. Queries are ranked by confidence and only the top p% are answered.
The mirage shows up on every architecture and gets worse as reasoning gets harder. On BRIGHT, every model's MaxScore AUROC sits in 0.520–0.611, which is not an actionable abstention signal. On TEMPO, Qwen falls to 0.536 and ReasonIR to 0.544. On BEIR, BGE and SFR still reach 0.749 and 0.754; ReasonIR is already at 0.621.
Distributional signals recover a usable slice of calibration:
| Model | Task | MaxScore | Best distributional |
| Instructor-Large | BEIR | 0.583 | LSMV 0.741 (+0.158) |
| Qwen | TEMPO | 0.536 | Gap 0.661 (+0.125) |
| Qwen | BRIGHT | 0.571 | Gap 0.653 (+0.082) |
Variance-based metrics differ from each other by at most about 0.017 AUROC, while beating MaxScore by 0.05–0.16. The paper frames that as a 5–10× gap: dropping magnitude matters far more than which spread statistic you pick. Sparse retrieval is the exception: on TEMPO, BM25's NQC (0.602) clearly beats Score Gap (0.526).
Continuous correlations line up the same way. For Qwen at k=25 on TEMPO, MaxScore's Pearson r with NDCG is 0.028 against Score Gap's 0.303; on BRIGHT the pair is 0.193 vs 0.401.
Downstream, the no-abstention baseline is 63.5% accuracy. MaxScore at 10% coverage reaches only 65.8%, and several retrievers fall below the baseline. Gap@25 scores 74.1 / 71.0 / 68.8 at 10 / 25 / 50% coverage. The sharpest single-retriever case is Qwen2 at 10% coverage: MaxScore keeps queries that are answered correctly 39.1% of the time, 26 points under the 65.8% baseline; Gap@25 keeps 76.8%. Same queries, same retriever, 38-point swing. For Qwen2, the 95% bootstrap CI on MaxScore AUROC for answer correctness is [0.398, 0.459], strictly below chance: raw similarity is anti-correlated with being right.
This is a config change, not a new model. The top-k scores already exist; swapping the threshold to Score Gap or LSMV adds no extra inference, no retraining, no latency. The paper itself calls the recovered signal meaningful but moderate: the best BRIGHT AUROC is still around 0.65. The result argues against using s1. It does not argue that abstention is now solved.
For teams already shipping a fixed similarity cutoff, this is cheaper than an LLM judge or a cross-encoder by a wide margin. Thresholds still need to be calibrated on a held-out set to a target precision or abstention rate.
The end-to-end study uses one generator (Qwen3.6-27B) and one judge (GPT-4o). Whether the gains survive a different model, prompt, or grading protocol is left open. All numbers are on public benchmarks; domain corpora, proprietary query mixes, and in-house retrievers may calibrate differently.
Binary success is NDCG@k > 0, which only asks whether one relevant document appeared. Appendix tables keep the same method ranking from k=5 to k=50, but the absolute numbers get looser as k grows.
The paper never pins the ceiling for zero-cost abstention. How much of the remaining gap a cross-encoder or an LLM verifier would close is unmeasured. Score Gap loses to MaxScore on 2 of 12 BRIGHT subsets; those reversals are at most 0.055 AUROC.