Discussions Heat Up Over KDA Linear Architecture
teortaxesTex · x · 2026-07-18
The author argues that KDA is a "good design," pushing back against the surprise surrounding it: the Kimi team turning it into a product-level result inherently proves extensive research. Furthermore, the 48B version already performed well, making the KDA route entirely unsurprising.
The embedded images focus on the Kimi Linear report and subsequent replication experiments. Some compared KDA against full attention and sliding window attention, finding it performed strongly on tasks like MOAR. Another image summarized the engineering hurdles of KDA compared to full attention, such as implementation, hyperparameter tuning, parallelism, and aligning training/inference numerics. The overarching takeaway is that this linear/alternative attention scheme isn't a "half-baked new architecture," but a robust solution standing on rigorous experimentation and engineering refinement.
More from Models
- OpenAI lists GPT-6.1 Terra: 1.05M context, $1/$5 pricing — but this poster sticks with Opus 5.5 — BLCNYY · 2026-10-27
- GPT-6.1 Terra cuts factual error rate from 22.3% to 4.3%, an ~81% drop — haider1 · 2026-10-27
- GPT-6.1 Terra lands: early users praise speed, value and subagent performance — BenBajarin · 2026-10-27
- Leaked GPT-6.1 Terra benchmarks: cheap but 'boring', still far behind Astra and Claude — Angaisb_ · 2026-10-27
- Artificial Analysis Ships Intelligence Index v4.3.2 With GPT-6.1 Terra Comparison — ArtificialAnlys · 2026-10-27
- GPT-6.1 Terra cuts hallucination rate by 50 points while lifting accuracy 2 points — ArtificialAnlys · 2026-10-27