SPARSEUP Recipe Tops BEIR-13 at 56.4 nDCG@10 for Sub-150M Sparse Retrievers
_reachsumit · x · 2026-11-10
Thibault Formal et al. present SPARSEUP, a recipe for training Splade-like sparse retrievers on modern backbones. Key contributions: analysis of how logit distribution affects training stability and sparsity; a new per-position fan-out hard cap limiting vocabulary dimensions each position may emit; and a vocabulary merging strategy that removes redundant case/word-boundary variants from cased BPE backbones. Applied to the LateOn backbone, SPARSEUP reaches 56.4 average nDCG@10 on BEIR-13 — the highest known among public vocabulary-based sparse encoders under 150M parameters. Paper and model are open on arXiv and Hugging Face.
Related event: Linkup's SPARSEUP Paper Sets New BEIR Record for Sparse Retrievers(2 posts)→
More from Research
- Base log perplexity is an extremely strong predictor of Epoch and OpenAI ECI, researchers note — stochasticchasm · 2026-11-23
- ECHO-style SFT losses in RL dodge the elicitation-only criticism, researcher argues — stochasticchasm · 2026-11-23
- Idea: humans write high-level proofs, AI fills in lemma details in Lean — theshawwn · 2026-11-23
- Claude points to Lean blueprints, Lamport structured proofs, and Gödel's speedup theorem — theshawwn · 2026-11-23
- Two-person lab GSI claims 250+ breakthroughs across 8 fields with custom AI-built solvers, not brute-force compute — udmrzn · 2026-11-23
- Geoffrey Irving: statistical physics offers a route to obfuscated arguments in scalable oversight — AndrewCritchPhD · 2026-11-23