LightOn open-sources SlideOn sparse retrieval models that beat their dense baseline on BEIR
tomaarsen · x · 2026-10-20
LightOn released SlideOn (174M, English) and mSlideOn (339M, multilingual) sparse retrieval models, both integrated with sentence-transformers via SparseEncoder(..., trustremotecode=True).
Key points:
- They emit 32,768-dim non-negative sparse vectors scored by dot product, storable in an inverted index; dims are independent of vocabulary tokens
- Method: SAE pretraining plus supervised retrieval fine-tuning, starting from DenseOn (English) and mLateOn (multilingual, long-context, code)
- SlideOn scores 56.68 nDCG@10 on BEIR-15, beating its dense starting point DenseOn (56.20) and nearing LateOn (57.22)
- Matryoshka-style query/document pruning lets you tune sparsity at inference without retraining
- Full training data and code released: 16.3M query-positive pairs across nine languages plus code, with cross-encoder teacher scores
Related event: LightOn Fully Open-Sources Sparse Retrievers SlideOn and mSlideOn(26 posts)→
More from Models
- Fable 5.5 leak: litellm code shows Anthropic model in staging, launch tipped for Oct 27 — imjustnewatai · 2026-10-21
- Open models below 4-bit struggle at self-iterating 3D asset generation, HF dev finds — mervenoyann · 2026-10-21
- Researcher Finds Two Windows 0-Days, First in 76 Minutes with a Frontier Model — moyix · 2026-10-21
- Dev finds Gemini Argon's debug-log analysis beats Fable 5.1 — ShikharMurty · 2026-10-21
- ChatGPT Gets Stuck Screaming "AAHH" in an Infinite Loop After a Simple Prompt — AutomaticScene · 2026-10-21
- Open-weights Ornith-1.5-122B debuts with 84.5% SWE-bench Verified, runs on a single DGX card — _akhaliq · 2026-10-21