Lychee-Reranker: decoupled FBNL reranker up to 4B params, strong on BEIR/MIRACL
tomaarsen · x · 2026-10-30
tomaarsen highlights the Lychee-Reranker paper (arXiv 2606.22807, Xinping Zhao et al.), noting its training combines supervised fine-tuning, two distillation branches, and model soup (averaging checkpoints into the final model).
Paper highlights:
- FBNL architecture: fast but not late-interaction, decoupling query and passage computation while keeping fine-grained relevance modeling.
- Design: encoder-decoder with Matryoshka embedding pooling for offline passage pre-encoding; the decoder models system/user instructions with query intent, and cross-attention captures fine-grained relevance between query context and passage representations.
- Four advantages: efficiency from offline passage encoding, expressiveness from cross-attention, compactness from Matryoshka pooling, and test-time compute via an adjustable compute budget.
- Three sizes: Nano/Small/Large with 0.27B/1B/4B activated parameters; strong results on BEIR, MIRACL, and LMEB.
More from Research
- Physicist calls OpenAI's Navier-Stokes solution physically meaningless despite loophole win — GaryMarcus · 2026-10-30
- ELLIS x UniReps talk: why distinct learning processes yield similar representations — ClementineDomi6 · 2026-10-30
- Anthropic paper: 0.1% pretraining poison plants RCE backdoor in 4B CLI agent — omarsar0 · 2026-10-30
- CopeNLU presents 7 papers at EMNLP 2026 on bias, alignment, explainability — IAugenstein · 2026-10-30
- Two-stage 3D ResNet pipeline detects neonatal brain pathologies from MRI with 0.919 AUC — Dr_Alex_Crimi · 2026-10-30
- 40 years of putting new neurons into the CNS: why so slow, and can AI accelerate it? — patrickmineault · 2026-10-30