307M Parameter Model mLateOn Excels in Multilingual, Long Doc, and Code Retrieval
antoine_chaffin · x · 2026-07-30
The author introduces a new 307M parameter retrieval model that delivers strong performance across English, multilingual, long-document, and code retrieval tasks.
Key Highlights & Ablations:
- Code Retrieval: Despite no code-specific pre-training, fine-tuning on the LateOn-Code dataset yields strong scores of 71.5 (mDenseOn) and 73.5 (mLateOn) on the MTEB Code benchmark.
- Data Efficiency: Ablation studies reveal that the model only requires a small amount of English/cross-lingual data to achieve robust generalization.
Related event: LightOn Open-Sources Multilingual Retrieval Models mDenseOn and mLateOn(18 posts)→
More from Research
- SimpleMark: multi-bit LLM watermark bounds per-token distortion with f-divergence budget — rishabh16_ · 2026-10-16
- Stanford paper: generative representation learning cuts tokenizer training compute by 7-8x — HirokatuKataoka · 2026-10-16
- Cohere Labs to present confidence estimation for reasoning models at EMNLP 2026 — Cohere_Labs · 2026-10-16
- Vals RSI Index extrapolates Google models hitting 130% autonomous AI R&D by Oct 2027 — HaydnBelfield · 2026-10-16
- SpatialClaw Pulled Into Agentic Vision Data; Filtering and Evals Were the Real Lift — CMHungSteven · 2026-10-16
- Visual Lean: drag-and-drop GUI for the Natural Numbers Game runs Lean 4 in WASM — MilesCranmer · 2026-10-16