LightOn Open-Sources Multilingual Retrieval Models mDenseOn and mLateOn

LightOn released fully open-source 307M-parameter multilingual retrieval models mDenseOn and mLateOn, focusing on multilingual, long-context, and code retrieval. The release includes model weights, 2.8B training pairs, 16.3M fine-tuning samples, and complete training code. The models achieve state-of-the-art results on BEIR, MLDR, MIRACL, and MTEB Code benchmarks, aiming to bridge the reproducibility gap caused by closed-source data.

Confirmed

• Models have 307M parameters and support 9 natural languages plus code retrieval.
• Open-sourced assets include model weights, 2.8B training pairs, 16.3M fine-tuning samples, and training code.
• Achieved SOTA on BEIR, MLDR, MIRACL, and MTEB Code benchmarks.
• Researchers cleaned and reconstructed 665M English contrastive pretraining pairs from 34 public sources, then translated them into 8 languages to build a 2.8B-pair pretraining set.
• Despite no dedicated code pretraining, fine-tuning on self-built LateOn-Code data yielded excellent code retrieval performance.
• Models are compatible with packed-encoders, enabling up to 5x training speedup on batches with many padding tokens.

Unconfirmed

• @antoinechaffin noted that despite mDenseOn's strong performance, the performance gap between late-interaction and dense models on long texts (e.g., MLDR) and languages outside the training setup remains to be further observed and evaluated.

Why it matters

• This release provides one of the largest open-source multilingual retrieval training sets. @reachsumit believes the approach aims to eliminate the reproducibility gap caused by closed-source data in current SOTA models, and @tomaarsen also emphasized the rarity of such training data scale.

2026-07-30 ~ 2026-07-31 · 18 related posts

• LightOn Open-Sources 149M Retrieval Models, Setting New BEIR SOTA — _reachsumit · 2026-07-30
• [source] LightOn Releases Fully Open Multilingual Retrieval Models mDenseOn & mLateOn — antoine_chaffin · 2026-07-30
• 307M Parameter Model mLateOn Excels in Multilingual, Long Doc, and Code Retrieval — antoine_chaffin · 2026-07-30
• Multilingual Retrieval Breakthrough: 307M Parameter Model Pushes Pareto Frontier — IgorCarron · 2026-07-31
• New Open Multilingual Retrieval Models Launch with Up to 5x Training Speedup — antoine_chaffin · 2026-07-31
• LightOn Open-Sources 149M Param Retrieval Models Achieving SOTA — antoine_chaffin · 2026-07-31
• LightOn Releases Strong Multilingual Retrieval Models with Zero-Shot Generalization — antoine_chaffin · 2026-07-31

11 near-duplicate retellings: antoine_chaffin · antoine_chaffin · tomaarsen · antoine_chaffin · IgorCarron · antoine_chaffin · antoine_chaffin · antoine_chaffin · antoine_chaffin · antoine_chaffin · IgorCarron