LightOn Fully Open-Sources Multilingual Retrieval Models and 2.8B Training Pairs

The LightOn team has released mDenseOn and mLateOn, 307M-parameter multilingual retrieval models. This is a highly comprehensive open-source release, including not only model weights but also 2.8 billion training pairs, 16.3 million fine-tuning samples, and the training code. The models achieve SOTA within their tier on benchmarks like BEIR and MLDR, aiming to bridge the reproducibility gap caused by closed-source data.

Confirmed

• The models feature 307M parameters and support retrieval across 9 natural languages and code.
• Open-source assets comprise model weights, 2.8 billion pre-training data pairs, 16.3 million fine-tuning samples, and full training code.
• Achieves SOTA within its class on BEIR and MLDR benchmarks, with excellent performance on MIRACL and MTEB Code.
• Researchers cleaned and reconstructed 665 million English contrastive pre-training data pairs from 34 public sources, translating them into 8 languages to build the 2.8 billion pair pre-training set.

Why it matters

• This release provides one of the largest open-source multilingual retrieval training sets to date. @reachsumit believes the initiative aims to eliminate the reproducibility gap in current SOTA models caused by their reliance on closed-source data, while @tomaarsen also highlighted the scarcity of training data at this scale.

2026-07-30 ~ 2026-07-30 · 11 related posts

• LightOn Open-Sources 149M Retrieval Models, Setting New BEIR SOTA — _reachsumit · 2026-07-30
• [source] LightOn Releases Fully Open Multilingual Retrieval Models mDenseOn & mLateOn — antoine_chaffin · 2026-07-30
• 307M Parameter Model mLateOn Excels in Multilingual, Long Doc, and Code Retrieval — antoine_chaffin · 2026-07-30

8 near-duplicate retellings: antoine_chaffin · antoine_chaffin · tomaarsen · antoine_chaffin · IgorCarron · antoine_chaffin · antoine_chaffin · antoine_chaffin