IFM releases K2-Horizon-MoVA-36B-A4B: 4B active params with 512K context
jacek2023 · reddit · 2026-09-03
IFM has released the final GGUF checkpoint of K2-Horizon-MoVA-36B-A4B, a sparse MoE model with Mixture-of-Values (MoVA) attention that stores 36B parameters but activates only 4B per token.
- Outscores open-weight dense (30B) and MoE models up to 15x its size on agentic and reasoning benchmarks, and competes with closed frontier models.
- Native 524,288-token context from midtraining onward.
- Fully open: training data/recipe, training code, and intermediate checkpoints will be released to study capability growth across training.
Related event: MBZUAI's IFM Releases K2 Horizon: Six Fully Open Models from 0.9B to 375B(30 posts)→
More from Models
- Users want open-format export from Claude Docs as charts only save as images — BlackHC · 2026-10-20
- EMNLP study: LLM safety interventions drop up to 4.5x when distress is wrapped in delusion — MaartenSap · 2026-10-20
- Cybersecurity-focused VLM OrcaCyber-Nano-1.0 trends on Hugging Face — orcarouter · 2026-10-20
- 0.8B local model with 2,000 labels beats Jev's frontier API on banking data — kalyan_kpl · 2026-10-20
- New TTS benchmark tests 26 models across 30,564 requests: fastest at 46ms — mattturck · 2026-10-20
- Agents handle unreleased language Jai flawlessly, even compile-time execution — lucasmeijer · 2026-10-20