IFM's K2-Horizon: six open models up to 375B MoE, with data and training code
winer666 · reddit · 2026-11-18
IFM (the institute behind MBZUAI, Abu Dhabi) released K2-Horizon: six open models from a 0.9B dense model to a 375B-A23B MoE flagship. The standout is the openness package — alongside weights: pretraining corpus, stage checkpoints, training logs, eval code, the xLLM training stack, and RL360 post-training code, framed as "glass box, not black box." Four models share a 22T-token corpus, with 10T synthetic pretraining tokens documented.
Technical notes from the paper (arXiv 2611.21003):
- MoVA (Mixture-of-Value Attention): MoE-style routing on attention value projections; the 36B-A4B MoVA model nearly matches a 32B dense model trained on the same data while activating 1/7 the parameters.
- Uno: a lossless diffusion adapter enabling up to 2.2x faster decoding without a separate draft model.
- The 375B flagship reportedly tops the Toolathlon tool-use benchmark (vendor-reported).
Caveats: a 375B MoE is infra news, not a laptop download; benchmark claims need independent runs; Abu Dhabi origin means checking data-residency/export rules. Available via vLLM, SGLang, Ollama. The open question: does shipping data and training code redefine "open" for the next frontier releases?
Related event: IFM Open-Sources K2-Horizon Fleet up to 375B(3 posts)→
More from Models
- Qwen 3.8 27B vs Flash-Next vs DeepSeek V4.1 Flash: one-shot landing page build-off — jesdga95 · 2026-11-18
- Halocline inference engine hits 148-182 tok/s on Qwen-Flash-Next with KLD 0.082 — Distinct-Pie2389 · 2026-11-18
- Domingos on iWSP paper: just use the program instead of writing it into LLM weights — pmddomingos · 2026-11-18
- GPT-6.2 Astra and Sol post highest error rates ever on VoxelBench 2 — legit_api · 2026-11-18
- GPT-6.2 Astra hits new harmonic milestones on the Bach Benchmark — deanwball · 2026-11-18
- OpenAI: GPT-6.2 Astra is first external model to hit "High" AI self-improvement — ChrisGPT · 2026-11-18