K2-Horizon-36B-A4B Scores 25 on AA Index With Only 4B Active Params via New MoVA Architecture
Aiden_Tech_Ai · x · 2026-09-22
K2-Horizon-36B-A4B packs 36B total parameters with 4B active per token, scoring 25 on the Artificial Analysis Intelligence Index — matching models with over 20× its total parameter count.
The gain comes from MoVA (Mixture-of-Value Attention), a new architecture that brings MoE-style sparsity into the value-vector computation of multi-head attention, opening a second axis for scaling sparsity beyond FFN-level MoE. Claimed advantages:
- Simple and compatible with efficient attention methods (flash attention, GQA, sparse attention)
- No additional KV cache cost compared to standard GQA
The model is now available.
More from Models
- OpenAI's five biggest failures laid out: pretraining decline, alignment fears, enterprise lost to Claude — teortaxesTex · 2026-11-12
- Fable 5.2 Hits 41.3% on Remote Labor Index, Up From 2.5% a Year Ago — alliekmiller · 2026-11-12
- Beaker automates MIT's weight-space programming: one-neuron edit lifts MATH-500 from 7% to 37% for under $4 — VictorSanh · 2026-11-12
- Researcher says Anthropic dodged questions on Mythos false positive rate; 37,966 findings, 2 exploited — OwariDa · 2026-11-12
- Grok spotted building meeting "Listen to Conversation" feature with live transcript — nima_owji · 2026-11-12
- MoonMath ships uncensored 763B DeepSeek-V4.1 via closed-form weight editing, claims zero capability loss — udmrzn · 2026-11-12