LightOn open-sources SlideOn sparse retrievers: 174M/339M params, tops BEIR-15 at 56.68
IgorCarron · x · 2026-10-19
LightOn released SlideOn (174M, English) and mSlideOn (339M, multilingual), two open sparse retrieval models built via SAE pretraining + supervised fine-tuning on top of their own DenseOn/mLateOn to learn sparse representations compatible with inverted-index search:
- SlideOn hits 56.68 on BEIR-15, beating DenseOn's 56.20, leading the search/storage efficiency Pareto frontier among sub-2B models
- mSlideOn beats mDenseOn by +4.76 on multilingual retrieval (MIRACL) and +10.19 on long-document retrieval (MLDR), also surpassing mLateOn and mDenseOn on code retrieval (MTEB Code)
- Matryoshka-style training keeps performance stable as pruning increases, letting users tune for latency/storage budgets
Starting from trained dense/late-interaction retrievers is far cheaper than redoing large-scale retrieval pretraining, and it improved mSlideOn's generalization to unseen languages and long context. SAE checkpoints, fine-tuned models and training code are released under permissive licenses.
Related event: LightOn Open-Sources SOTA Sparse Retrievers SlideOn and mSlideOn(6 posts)→
More from Models
- Open-weights Muse Spark on B300 costs $0.12 vs $0.88 per correct task, 7.1x cheaper — ryanshrout · 2026-10-20
- Mistral launches Timestral time-series foundation models, zero-shot SOTA on GIFT-Eval — victormustar · 2026-10-20
- Sabi open-sources 27B BCI model with 15% hallucination rate, far below GPT-6 — lukas_m_ziegler · 2026-10-20
- MiniMax-M3.1-Flash scores 42 on AA Intelligence Index, halving cost per task to $0.22 — ArtificialAnlys · 2026-10-19
- DeepSeek V4.1 Flash 1017 lands on HuggingChat with free unlimited Exa web search and HF MCP — victormustar · 2026-10-19
- 221 fresh chats: Claude Sonnet 5.5 says "47" for 92% of random number requests — randal_olson · 2026-10-19