DecisionRank: 1.7B reranker beats its 27B teacher, runs 24.5x faster at 8B
kalyan_kpl · x · 2026-11-02
DecisionRank trains decision-model rerankers via item-space on-policy distillation (OPD): the student's document scores are sorted into a ranked sequence, and a generative teacher provides conditional feedback on that list to directly supervise the scores.
Results:
- SoTA on general and reasoning-intensive retrieval benchmarks including TREC DL, BEIR, and BRIGHT
- Item-space OPD consistently outperforms alternative training objectives across four benchmarks
- At just 1.7B parameters, DecisionRank surpasses its 27B generative teacher on TREC DL19–23
- The 8B version beats its generative listwise counterpart while running 24.5x faster on TREC DL and 16.7x faster on BRIGHT
Related event: Meta's DecisionRank: 1.7B Distilled Reranker Beats 27B Generative Teacher(3 posts)→
More from Research
- Anthropic Open-Sources FlashProve, Zero-Knowledge Proofs of AI Inference, Tested on Llama 4 Maverick — yenkel · 2026-11-03
- BiGym 2.0: Claude-written policies hit 51.9% on humanoid household tasks, nearing VLA — teortaxesTex · 2026-11-03
- Hopfield model thermodynamic limit proven at all loads without replica-symmetry assumptions — _onionesque · 2026-11-03
- 3D World Simulator debuts: explicit 3D representation for long-horizon rollouts — Scobleizer · 2026-11-03
- Goertzel Weighs Limits of Weight-Editing RSI: Target Nets 10,000x Smaller, New Functions Hard — bengoertzel · 2026-11-03
- TTCA metric and LAAR routing: measuring time until the first correct answer, not just latency — burkov · 2026-11-03