Translated open-sources Lara Mini Base: 0.6B-active MoE beats models trained on 4-6x more tokens
NielsRogge · x · 2026-11-16
Translation company Translated released Lara Mini Base, an open-weights model built around the thesis that language diversity is a resource, not a trade-off. The paper notes non-English data makes up at most 12% of pretraining in the open models they reviewed.
Key facts:
- MoE with 7.2B total / 0.6B active parameters, trained from scratch on 6T tokens in English plus 44 other languages, covering all 24 official EU languages
- Highest average score among open base models with up to 2B active parameters across English, multilingual, math and code benchmarks, beating models trained on 4-6x as many tokens; biggest margins on multilingual and math benchmarks
- 128k context support; generates text at least 1.8x faster on a single GPU than any higher-scoring model, without quantization
- Weights released under CC BY-NC-SA 4.0; team will also release T-HellaSwag, a machine translation of HellaSwag into 110 language variants
The authors find the MoE architecture reduces capacity competition between languages — multilingual training can help rather than dilute small models.
More from Models
- Fine-tuned 68M model beats Claude Opus 5 on bounded tasks at 12ms on CPU — sudalairajkumar · 2026-11-16
- Gemini 4 Hydrogen launched: 2.5x faster than 3.8 Flash, but how much is batching? — krishnan · 2026-11-16
- X User Flags Likely AI-Slop Blog That Repackages Standard Model Ladder Training as Novel 'Laddering' — code_star · 2026-11-16
- 'Alignment of the Gaps': Higher Eval Scores Don't Mean Newer Models Are Truly More Aligned — gleech · 2026-11-16
- 'It Knows Google Is Watching': Viral Chat Surfaces J-Lens Evaluation-Awareness Research — repligate · 2026-11-16
- Tencent Hunyuan launches Hy4 flash: matches Claude Haiku 5.5 at 1/18th the cost — TencentHunyuan · 2026-11-16