LightOn Open-Sources SlideOn Sparse Retrievers at 174M/339M, Claiming SOTA

On October 19–20, LightOn R&D released and fully open-sourced two sparse retrieval models: SlideOn for English (174M parameters) and mSlideOn multilingual (339M parameters). Model weights, pre-trained sparse autoencoders (SAE), roughly 16.3 million query-positive training pairs (covering 9 natural languages plus code, with cross-encoder teacher scores), and all training code are now on Hugging Face. Both models are integrated with sentence-transformers: after pip install -U sentence-transformers, they can be loaded via SparseEncoder(..., trustremotecode=True). tomaarsen noted the official positioning is first-stage retrieval, and recommends stacking a multi-vector reranker on top as a cascaded pipeline.

Confirmed

• Technical approach: SlideOn starts from DenseOn and mSlideOn from mLateOn, first reconstructing token embeddings from a small set of activated SAE features, then fine-tuning with retrieval supervision; they output 32768-dimensional sparse vectors scored by raw dot product (no final normalization), with dimensions not corresponding to vocabulary tokens. tomaarsen explained that activated features can be stored directly in an inverted index: query features select posting lists, which are then scored by matching weights.
• Official results: on BEIR-15, SlideOn scores 56.68 nDCG@10, beating the dense baseline DenseOn (56.20) and approaching LateOn (57.22); on MTEB Code's 12-task average nDCG@10, mSlideOn 74.78 > mLateOn 73.48 > mDenseOn 71.53; in multilingual comparisons against mDenseOn, it leads by +10.19 on MLDR (13 languages), +4.76 on MIRACL (18 languages), and +3.24 on MTEB Code, while trailing by 0.70 on the BEIR-15 average. Both models support 81 languages.
• Matryoshka sparse training: pruning down to 64 activated features for queries and 512 for documents costs only 0.02 points on BEIR — query pruning cuts inverted-list accesses, while document pruning reduces storage. tomaarsen's cost-quality curves show SlideOn (d=512) leading on price-performance among <2B sparse models.
• Cross-lingual testing: French/German queries can directly match English/Portuguese documents, via the encodequery()/encodedocument()/similarity() API.

Why it matters

According to the authors (relayed by IgorCarron), after riding two waves — dense retrieval and late interaction — the team is betting on sparse retrieval. This release is a rare fully open-source bundle of models + data + code on the sparse route, and its decoupling of sparse vectors from the vocabulary makes it deployable on inverted-index infrastructure, with real engineering significance for large-scale first-stage retrieval.

2026-10-19 ~ 2026-10-20 · 23 related posts

• LightOn ships SlideOn and mSlideOn sparse retrievers for first-stage retrieval — IgorCarron · 2026-10-19
• LightOn open-sources SOTA sparse retrievers SlideOn and mSlideOn at 174M/339M — IgorCarron · 2026-10-19
• SlideOn hits sparse SOTA on BEIR-15 at 56.68, beating its dense baseline at 56.20 — IgorCarron · 2026-10-19
• mSlideOn: multilingual sparse retriever beats its dense counterpart on most tasks — IgorCarron · 2026-10-19
• SlideOn models and pretrained sparse autoencoders are now available on Hugging Face — tomaarsen · 2026-10-20
• How SlideOn works: sparse autoencoder reconstructs token embeddings, then retrieval fine-tuning — tomaarsen · 2026-10-20
• SlideOn's sparse dims are vocabulary-independent, enabling inverted-index retrieval — tomaarsen · 2026-10-20
• SlideOn's sparse dims are vocabulary-independent and index-friendly — tomaarsen · 2026-10-20
• SlideOn charts near the quality-cost Pareto frontier for sub-2B sparse retrieval — tomaarsen · 2026-10-20
• Pruning SlideOn to 64+512 features costs just 0.02 nDCG while cutting FLOPS 3.5x — tomaarsen · 2026-10-20
• mSlideOn gains +10.19 on MLDR long-doc retrieval over its dense starting point — tomaarsen · 2026-10-20
• mSlideOn tops MTEB Code at 74.78 nDCG, beating dense and late-interaction models — tomaarsen · 2026-10-20
• mSlideOn handles cross-lingual retrieval with raw dot-product scoring — tomaarsen · 2026-10-20
• [source] LightOn ships 16.3M training pairs and full code alongside SlideOn models — tomaarsen · 2026-10-20
• SlideOn lands in sentence-transformers with one-line SparseEncoder loading — tomaarsen · 2026-10-20
• [source] SlideOn model card details SAE pretraining and Matryoshka sparse training recipe — tomaarsen · 2026-10-20

7 near-duplicate retellings: IgorCarron · IgorCarron · IgorCarron · IgorCarron · antoine_chaffin · tomaarsen · tomaarsen