IFM's K2-Horizon-36B-A4B Matches 20x-Larger Models on AA Index Using New MoVA Architecture
victormustar · x · 2026-09-22
IFM has open-sourced K2-Horizon-36B-A4B, which scores 25 on the Artificial Analysis Intelligence Index while activating only 4B parameters per token — matching models with over 20x its total parameters.
The gains come from MoVA (Mixture-of-Value Attention), a new architecture that applies MoE-style sparsity to the value vectors in multi-head attention, opening a second sparsity scaling axis beyond FFN-level MoE:
- Simple and compatible with efficient attention methods (FlashAttention, GQA, sparse attention)
- No extra KV cache cost compared to standard GQA
The model is available on Hugging Face as IFM/K2-Horizon-MoVA-36B-A4B.
More from Models
- HackerGLM games the grader: guesses 11 hidden test IDs to pass a buggy RL environment — echen · 2026-11-12
- ADM 3 Cloud flops in image evals: ranks #86 of 107 in text-to-image — ArtificialAnlys · 2026-11-12
- YC-backed Monte launches post-training platform to build proprietary models from your agents' real work — ycombinator · 2026-11-12
- Leak Claims OpenAI Will Launch Entire GPT-6.2 Family Tomorrow to Counter Opus 5.5 and Fable 5.2 — heypearlai · 2026-11-12
- ARC-AGI-3 leaderboard: models score 90%+ but at $10K+ per task — haider1 · 2026-11-12
- LMArena Launches Agent Arena for Head-to-Head Agent Evaluation — arena · 2026-11-12