IFM releases K2-Horizon: six fully open LLMs up to 375B MoE, trained on ~10T synthetic tokens
IFM · hf · 2026-11-17
IFM (MBZUAI's Institute of Foundation Models) released K2-Horizon, a fleet of six fully open language models from 0.9B dense to 375B-A23B MoE, leading or matching peer open models on agentic and coding benchmarks.
- MoVA applies MoE to attention value projections with softplus gating; the 36B-A4B model nearly matches a 32B dense model trained on the same data with 1/7 the active parameters.
- Uno diffusion adapters speed up decoding up to 2.2x losslessly without a draft model.
- Full LLM360-style release: intermediate checkpoints, pre-training data, logs, and all training/eval/infra code including the xLLM framework and RL360 RL stack, plus documented data pipelines producing 10T synthetic pre-training tokens.
- Four models share one 22T-token corpus and recipe, enabling controlled scaling studies.
All artifacts at ifm.ai/k2.
More from Models
- Reuters exclusive: China taps US AI models via distillation to train homegrown rivals — ns123abc · 2026-11-17
- Optimization researcher: Astra proved with one prompt what we couldn't — bremen79 · 2026-11-17
- Gemini's AI Overview caught fabricating brake rotor dimensions for a Fisker Ocean query — AWTom · 2026-11-17
- Dev Slams Claude for Building a Content Repository With No Way to Add Content — dejanseo · 2026-11-17
- You can't bypass AI detectors — a random word list still scores 100% AI on Pangram — rubenhassid · 2026-11-17
- Tencent Hunyuan highlights user demo: Hy4 Flash builds a stunning castle with only 75k tokens — TencentHunyuan · 2026-11-17