Study: reranker fine-tuning beats decision models by 5-29 nDCG points up to 9B params
kalyan_kpl · x · 2026-10-26
A University of Passau/IT:U study compares decision models (calibrated probability output) against reranker fine-tunes across six Qwen backbones (0.6B-27B) on TREC DL and BEIR collections. Reranker fine-tuning wins by 5-29 nDCG@10 points on every backbone up to 9B, including zero-shot; they tie at 27B. Below 9B, decision fine-tuning adds nothing over zero-shot prompting, and its calibration advantage only holds for certain question phrasings.
Related event: Paper Shows Decision Models Fall Short as Rerankers(2 posts)→
More from Research
- Anonymous Sleuth Shows Nature fMRI Foundation Model Paper Data Was Random, Seed 42 — bttyeo · 2026-10-26
- Learning RL From Scratch: Dev Publishes One-Page Basics Notes on neatrl.com — East-Muffin-6472 · 2026-10-26
- Token embeddings 'looked for' by value matrices but contribute little, analysis finds — RichmanRonald · 2026-10-26
- 'Neural optical flow': parameterizing classic CV circuits for faster network convergence — mike64_t · 2026-10-26
- Biomedical AI veteran: 'solving' biology is limited by data, not compute — iScienceLuvr · 2026-10-26
- New paper: training tasks act as attractors, clustering network weights by dataset — wschroll · 2026-10-26