LightOn releases SlideOn & mSlideOn: 174M English and 339M multilingual sparse retrieval models
tomaarsen · x · 2026-10-20
LightOn shipped SlideOn and mSlideOn, 174M English and 339M multilingual sparse retrieval models that work with Sentence Transformers, with adjustable query & document sparsity; mSlideOn also handles long documents and code.
SlideOn initializes from DenseOn and mSlideOn from mLateOn. A sparse autoencoder first reconstructs token embeddings using a few active features, then the backbone and sparse encoder are fine-tuned for retrieval — helped by base models already being adept at retrieval.
Both output 32,768-dim sparse vectors scored by dot product, with dimensions independent of vocabulary tokens: active features can live in an inverted index, where query features select posting lists and matching weights contribute to scores. Some learned dimensions respond to recognizable concepts like "solar power" or "batteries". Pretrained SAEs and models are released; the blog covers method, results and ablations.
Related event: LightOn Open-Sources SOTA Sparse Retrievers SlideOn and mSlideOn(26 posts)→
More from Models
- OpenAI API adds 12-hour prompt caching option alongside 30-minute default — OpenAIDevs · 2026-10-21
- antirez on model upgrades: 4.0→4.1 knowledge transfers unless attention changes — teortaxesTex · 2026-10-21
- DeepSeek reportedly profits at $0.15/$0.60 off-peak pricing — the price Western labs must beat — arthurcolle · 2026-10-21
- SpecterOps Researchers Show Adversarial ML Evading EDR, LLMs Speeding Reverse Engineering — dyn___ · 2026-10-21
- AWS fine-tunes GPT-OSS-20B for multilingual reasoning: base model reasoned in French 0% of the time — AWS ML Blog · 2026-10-21
- Riemann-bench: frontier models jump from under 5% to over 70% in six months, NS and UGC solved — echen · 2026-10-21