Meta's DecisionRank: 1.7B Reranker Beats 27B Teacher While Running 24.5x Faster
_reachsumit · x · 2026-11-02
Meta published the DecisionRank paper introducing on-policy distillation for decision-model rerankers.
- Decision models predict choices or scores directly, avoiding autoregressive decoding latency, but transferring a generative teacher's ranking knowledge is hard: the teacher expresses preferences as ranked document ID sequences while the student scores via the first-token position
- Item-space on-policy distillation (OPD) constructs the student's rollout by sorting its scores into a ranked list, which the generative teacher then supervises with conditional feedback, combined with offline ranking supervision
- Achieves state-of-the-art on TREC DL, BEIR, and BRIGHT; item-space OPD consistently outperforms alternative objectives across four benchmarks
- At 1.7B parameters, DecisionRank surpasses its 27B generative teacher on TREC DL19–23; at 8B it runs 24.5x faster on TREC DL and 16.7x faster on BRIGHT than its listwise counterpart
More from Research
- Mollick: expert-level work benchmarks are brutally hard, and ARC-AGI, GDPval, METR are saturated — emollick · 2026-11-02
- Rebuilding a Protein Domain Distribution Pipeline After a Backup Failure — AryHHAry · 2026-11-02
- NVIDIA's Elastic Latent Recurrence lets one 4B model trade loop count for accuracy: 18→29 on AA Index — burny_tech · 2026-11-02
- IBM's STAIR retriever cuts invalid references 65× and hits 82.6% accuracy by using document structure — solyarisoftware · 2026-11-02
- ReLift Hits SOTA Dense 3D Tracking on WorldTrack With 2.7x Faster Pipeline — Technion · 2026-11-02
- Why robotics "capability jumps" are just aligning tasks to pretrained LLM skills — robrombach · 2026-11-02