Is SWA the frontier secret sauce? DeepSeek and Kimi chose the hard independent route
burny_tech · x · 2026-10-17
X user teortaxesTex argues that sliding-window attention (SWA) is the real 'frontier secret sauce' — startups spun out of American frontier and near-frontier labs (e.g. Mistral) universally use it, while DeepSeek and Kimi went the hard independent R&D route: MHA→MLA→NSA→DSA (through whatever V4/V4.1 became), plus MoBA and Kimi-Linear. burnytech boosted it with a 'thonk'. The exchange highlights a real architectural split between frontier labs and the top Chinese open-source camps.
More from Models
- Meta drops 816B open-weight MoE, Anthropic hits $16.8B Q3, 8 states sue OpenAI — rohanpaul_ai · 2026-10-17
- Andriy Burkov: the only gap between open-weight and frontier models is data quality — burkov · 2026-10-17
- NVIDIA open-sources midtraining recipe: 500B tokens lift Nemotron agentic benchmark scores — denisyarats · 2026-10-17
- Liquid AI ships d1-ultra decision model, tops Artificial Analysis Decision Index at 65 — DeryaTR_ · 2026-10-17
- Quick take: Claude 5.5 Haiku trails 6.1 Luna, which roughly matches Gemini 3.8 Flash — burkov · 2026-10-17
- Anthropic launches Claude Haiku 5.5: 75% cheaper, 39.2% on Terminal-Bench vs GPT-6 Luna's 16.4% — dl_weekly · 2026-10-17