Magnitude Mirage: score-threshold RAG abstention fails on reasoning-heavy queries
_reachsumit · x · 2026-09-15
Across 11 retrievers and 28 datasets, raw similarity-score thresholding for RAG abstention degrades to near-random on logical and temporal reasoning queries — the 'Magnitude Mirage.' Studying six zero-cost QPP metrics across BEIR, BRIGHT, and TEMPO, the authors find score-distribution signals (Score Gap, LSMV) lift abstention AUROC by up to 0.16 with no extra inference or retraining.
More from Research
- DONIC: one policy drives two dexterous hands, 86.79% on unseen trajectories without fine-tuning — liuziwei7 · 2026-10-23
- Nature Neuroscience study: cortical network geometry shapes how task representations transform — pshrink · 2026-10-23
- Under matched token budgets, most gains of web agent online augmentation vanish — hllo_wrld · 2026-10-23
- ETH Zurich robot learns humanoid mobility from 13 minutes of human motion, no reference at deployment — ChongZzZhang · 2026-10-23
- DeepSeek-based agent harness tops Lean Eval at a fraction of the cost, talk reveals — lukaszkaiser · 2026-10-23
- SubSeq hands out $100 credits to celebrate RFD4-Proteina and BioEmu2 releases — AllThingsApx · 2026-10-23