SPARSEUP Recipe Tops BEIR-13 at 56.4 nDCG@10 for Sub-150M Sparse Retrievers

_reachsumit · x · 2026-11-10

Thibault Formal et al. present SPARSEUP, a recipe for training Splade-like sparse retrievers on modern backbones. Key contributions: analysis of how logit distribution affects training stability and sparsity; a new per-position fan-out hard cap limiting vocabulary dimensions each position may emit; and a vocabulary merging strategy that removes redundant case/word-boundary variants from cased BPE backbones. Applied to the LateOn backbone, SPARSEUP reaches 56.4 average nDCG@10 on BEIR-13 — the highest known among public vocabulary-based sparse encoders under 150M parameters. Paper and model are open on arXiv and Hugging Face.

Related event: Linkup's SPARSEUP Paper Sets New BEIR Record for Sparse Retrievers(2 posts)→

Original post →

More from Research

Research channel →