SlideOn: LightOn's Open Sparse Retrievers Top MTEB

Episode 1 · LightOn Open-Sources Sparse Retrievers SlideOn and mSlideOn, Beating Dense Baseline on BEIR (2026-10-19, 26 posts)

LightOn R&D released and fully open-sourced two sparse retrieval models on Oct 19-20: SlideOn (174M, English) and mSlideOn (339M, multilingual). Model weights, pretrained sparse autoencoders (SAE), 16.3M query-positive training pairs covering 9 natural languages plus code (with cross-encoder teacher scores), and all training code are on Hugging Face. Both work with sentence-transformers via SparseEncoder(..., trustremotecode=True); tomaarsen notes the intended role is first-stage retrieval, with a multi-vector reranker stacked on top.

Confirmed

• Method: SlideOn starts from DenseOn and mSlideOn from mLateOn; an SAE first reconstructs token embeddings with few active features (top-32 latent activations, ReLU, max-pooling across tokens), then retrieval-supervised fine-tuning. Both output 32,768-dim sparse vectors scored by raw dot product (no final normalization); dimensions do not correspond to vocabulary tokens. Per tomaarsen, active features can be stored in an inverted index: query features select posting lists, matched weights score.
• Results: SlideOn scores 56.68 nDCG@10 on BEIR-15, beating dense baseline DenseOn (56.20) and approaching LateOn (57.22). On MTEB Code (12 tasks): mSlideOn 74.78 > mLateOn 73.48 > mDenseOn 71.53, with mSlideOn beating all sparse and late-interaction baselines in LightOn's comparison. Versus mDenseOn: +10.19 on MLDR (13 languages), +4.76 on MIRACL (18 languages), +3.24 on MTEB Code, -0.70 on BEIR-15. Both support 81 languages; mSlideOn handles long-document retrieval.
• Matryoshka sparse training: pruning to 64 query / 512 document active features costs only 0.02 BEIR points; query pruning cuts posting-list access, document pruning cuts storage, with separate topk settings in sentence-transformers. tomaarsen's cost-quality curve shows SlideOn (d=512) leading value among <2B sparse models.
• Cross-lingual test: French/German queries directly match English/Portuguese documents via encodequery()/encodedocument()/similarity().

Why it matters

Per IgorCarron, the team bet on sparsity after riding the dense and late-interaction waves. This is a rare fully open model+data+code release for sparse retrieval; vocabulary-decoupled sparse vectors fit existing inverted-index infrastructure and run on CPU, reshaping enterprise retrieval cost structures without GPU vector search.

• LightOn ships SlideOn and mSlideOn sparse retrievers for first-stage retrieval — IgorCarron · 2026-10-19
• LightOn open-sources SOTA sparse retrievers SlideOn and mSlideOn at 174M/339M — IgorCarron · 2026-10-19
• SlideOn hits sparse SOTA on BEIR-15 at 56.68, beating its dense baseline at 56.20 — IgorCarron · 2026-10-19
• SlideOn hits the storage-efficiency frontier via Matryoshka training and vector pruning — IgorCarron · 2026-10-19
• mSlideOn: multilingual sparse retriever beats its dense counterpart on most tasks — IgorCarron · 2026-10-19
• LightOn open-sources SlideOn sparse retrievers: 174M/339M params, tops BEIR-15 at 56.68 — IgorCarron · 2026-10-19
• LightOn open-sources SOTA sparse retrievers SlideOn and mSlideOn at 174M and 339M parameters — IgorCarron · 2026-10-19
• LightOn open-sources SlideOn, SOTA sparse retrievers at just 174M and 339M params — IgorCarron · 2026-10-20
• LightOn open-sources SlideOn, SOTA sparse retrievers at 174M and 339M params — IgorCarron · 2026-10-20
• LightOn open-sources SOTA sparse retrievers SlideOn at 174M and 339M params — antoine_chaffin · 2026-10-20
• LightOn open-sources SlideOn sparse retrieval models that beat their dense baseline on BEIR — tomaarsen · 2026-10-20
• LightOn releases SlideOn & mSlideOn: 174M English and 339M multilingual sparse retrieval models — tomaarsen · 2026-10-20
• SlideOn models and pretrained sparse autoencoders are now available on Hugging Face — tomaarsen · 2026-10-20
• How SlideOn works: sparse autoencoder reconstructs token embeddings, then retrieval fine-tuning — tomaarsen · 2026-10-20
• SlideOn's sparse dims are vocabulary-independent, enabling inverted-index retrieval — tomaarsen · 2026-10-20
• SlideOn's sparse dims are vocabulary-independent and index-friendly — tomaarsen · 2026-10-20
• SlideOn charts near the quality-cost Pareto frontier for sub-2B sparse retrieval — tomaarsen · 2026-10-20
• Pruning SlideOn to 64+512 features costs just 0.02 nDCG while cutting FLOPS 3.5x — tomaarsen · 2026-10-20
• mSlideOn gains +10.19 on MLDR long-doc retrieval over its dense starting point — tomaarsen · 2026-10-20
• mSlideOn tops MTEB Code at 74.78 nDCG, beating dense and late-interaction models — tomaarsen · 2026-10-20

6 more related posts →

Episode 2 · LightOn's Open-Source SlideOn Models Top MTEB Retrieval Under 500M Parameters (2026-10-22, 3 posts)

LightOn open-sourced mSlideOn (Apache 2.0) and released the SlideOn series, which uses sparse autoencoders to convert dense retrieval models into sparse ones. SlideOn and mSlideOn took the top two spots in MTEB retrieval under 500M parameters.

• LightOn's SlideOn sweeps MTEB, taking top two spots for under-500M retrieval models — tomaarsen · 2026-10-22
• LightOn open-sources mSlideOn, best-ever sub-500M model on MTEB retrieval — tomaarsen · 2026-10-22
• LightOn open-sources SlideOn: SAE-based sparse retrieval beats its dense teacher at 174M params — tomaarsen · 2026-10-22