Linkup Open-Weights SparseUp: Strongest Sub-150M Sparse Retriever on BEIR
tomaarsen · x · 2026-09-17
Linkup released open-weight sparse embedding model SparseUp (Apache 2.0), praised by Sentence Transformers maintainer tomaarsen.
- Built on ModernBERT (149M), same backbone and fine-tuning data as LightOn's DenseOn/LateOn, completing the controlled three-architecture comparison
- Claims to be the strongest public vocabulary-based sparse encoder under 150M: 56.4 nDCG@10 on BEIR-13 (vs LateOn 57.9, DenseOn 56.9 under identical conditions)
- Three knobs vs vanilla SPLADE: logitshift=15 for sparse ReLU support, positiontopk=12 expansion budget per token, vocabfold collapsing case/space surface forms (vocab 50k → 34k)
- Contrastive fine-tuning on LightOn's mixture (7 hard negatives from 50 + in-batch), no cross-encoder distillation
Available on Hugging Face with inference-ready deployment.
Related event: Linkup Open-Sources SparseUp: 149M-Parameter Sparse Retriever Tops BEIR-13(12 posts)→
More from Models
- 'You Can Feel the Gradient Slopes': Dev Contrasts Benchmaxxed RL Models vs General Ones — theteknosaur · 2026-11-02
- LightOnOCR-3 Launches: Tops FRBench, Adds Layout Structure and Chart Extraction — IgorCarron · 2026-11-02
- NVIDIA's Kumo Is a Family of Four Tabular Foundation Models, Not Just One — pandeyparul · 2026-11-02
- SemiAnalysis: safety classifiers are underrated — Meta's Muse tops cyber benchmark at 35.2% — ThomasScialom · 2026-11-02
- DeepSeek V4.1 Flash 1017 wins over a longtime closed-source-only user: fast, cheap, reliable — _bachrc · 2026-11-02
- Local quants vs Claude on 9 real private bugs: 3-bit costs thinking time, not score — FeydRowan · 2026-11-02