Sparseup-fr: Replicating Linkup's Sparse Embedding Model in French on One GPU
tomaarsen · x · 2026-10-12
Community author Loïck Bourdois replicated Linkup's recently open-sourced 150M-parameter sparse embedding model (linkup-sparseup-embed-v1) as a French variant, Sparseup-fr.
Architecture changes: the English backbone was swapped for the multilingual mLateOn-unsupervised (covering 10 languages including French); the MLM head uses mmBERT-base instead of ModernBERT, with the vocabulary trimmed from 256,000 to 50,240 French-mined tokens — kept as a multiple of 64 for GPU efficiency while staying close to the English version's 50,370 tokens.
Training cost: 55 hours on a single RTX 6000 Pro. The original approach, based on LightOn's earlier work, grafts a ModernBERT MLM head onto LateOn-unsupervised and finetunes it, gaining 2 points on BEIR-13 over the prior SOTA of that model family.
Related event: French Sparse Retrieval Model sparseup-fr-v0 Released on Hugging Face(2 posts)→
More from Research
- Preparing post-training data for coding LLM fine-tunes is 'black art', says practitioner — ayushthakur0 · 2026-10-13
- JQD:DM launches Special Workflow Issue with shared skill-file LLM gatekeeping in peer review — manoelribeiro · 2026-10-13
- Depth vs hold: behavior alone can't diagnose why LLMs fail, new paper argues — burny_tech · 2026-10-13
- BudgetPix: adaptive tokenization matches full-compute pixel diffusion with 25% of the compute — _akhaliq · 2026-10-13
- SoftwareGym synthesizes GUI environments for computer-use agents, lifting OSWorld 37% relative — _akhaliq · 2026-10-13
- LightOn's open 2B model cuts Claude Code token bills by 34% via local semantic code search — lmoroney · 2026-10-13