Distilling decision-oriented LLM judgments into an MLP speeds up ICL demo retrieval
_reachsumit · x · 2026-11-04
Paper 2611.00737 shows Jev, a decision-oriented language model producing probabilistic judgments, can rerank in-context demonstrations by utility for a given query.
- Applying Jev at scale is costly since each candidate needs an extra inference pass
- The authors distill Jev's utility estimates into a lightweight MLP that ranks demonstrations efficiently while keeping competitive retrieval performance
More from Research
- Google and Mercor's PLE benchmark: all frontier agents score below 50% on 85 real engineering tasks — teortaxesTex · 2026-11-04
- Fable 5.1 hand-writes 100k-param classifier weights, beating backprop training — AccBalanced · 2026-11-04
- Cambridge's van der Schaar Lab opens PhD applications for 2027 entry — MihaelaVDS · 2026-11-04
- Tencent's Hyra 2.0 Generates Protein Sequences 57x Faster Than ProteinMPNN — YvesMulkers · 2026-11-04
- Researcher Urges Top AI Conferences to Desk-Reject AI-Generated Papers Amid Fake Data Scandals — Vegetable-Formal-753 · 2026-11-04
- Harvard Professors Admit AI Article Reviews Now Beat the Best Humans — iamfakhrealam · 2026-11-04