IFM open-sources K2 Horizon: MoVA architecture, 0.9B-375B, all weights and training data
solyarisoftware · x · 2026-09-04
IFM dropped K2 Horizon, six open models from 0.9B to 375B, possibly the most interesting open local release of the year:
- Flagship 36B-A4B: 36B total params, only 4B active per token, 512K native context, coding + agentic training, GGUFs appearing
- New MoVA architecture (Mixture-of-Value Attention): expert routing brought into attention, not just FFN — claimed to approach its dense 32B model
- Fully open: weights, training data/recipes, training code, intermediate checkpoints, fine-grained logs, agentic post-training recipes, Apache 2.0
- Full size range: 0.9B for watches/glasses, 3.7B for phones, 7B for local agents
Related event: MBZUAI's IFM Releases K2 Horizon: Six Fully Open Models from 0.9B to 375B(30 posts)→
More from Models
- Users want open-format export from Claude Docs as charts only save as images — BlackHC · 2026-10-20
- EMNLP study: LLM safety interventions drop up to 4.5x when distress is wrapped in delusion — MaartenSap · 2026-10-20
- Cybersecurity-focused VLM OrcaCyber-Nano-1.0 trends on Hugging Face — orcarouter · 2026-10-20
- 0.8B local model with 2,000 labels beats Jev's frontier API on banking data — kalyan_kpl · 2026-10-20
- New TTS benchmark tests 26 models across 30,564 requests: fastest at 46ms — mattturck · 2026-10-20
- Agents handle unreleased language Jai flawlessly, even compile-time execution — lucasmeijer · 2026-10-20