Chinese frontier models adopt linear attention; Zai releases MIT-licensed 320B GLM-5.3-Flash
ivan_bezdomny · x · 2026-08-26
Elie Bakouch observes that, except for DeepSeek and Kimi, all major Chinese frontier models have adopted linear attention and sparse attention designs with similar indexer/compression mechanisms. They also utilize advanced residuals (mHC, attention residual) and the Muon optimizer for efficiency. Concurrently, Zai released GLM-5.3-Flash (formerly Ox Alpha), a 320B-parameter (18B active) natively multimodal model with a 1M-token context window. It runs entirely on Chinese AI chips and is released under the MIT License.
Related event: Chinese Frontier Models Shift to Linear Attention and Open Source(2 posts)→
More from Models
- Nat Lambert: wave of AI releases rumored — thinky, muse spark, Fable 5.5, OpenAI — natolambert · 2026-10-15
- Musk amplifies dev's verdict: Grok Bot easier to use than Codex and Claude — elonmusk · 2026-10-15
- Rumor: New Anthropic model imminent, apparently behind a batch of high-severity Chrome CVEs — matthew_d_green · 2026-10-15
- StartLux-Decision debuts RL-trained calibrated decision models, 27B scores 72.8 — startlux-models · 2026-10-15
- Meta's Muse Code slammed for harsh quotas: 21% left after 5 hours — ThomasScialom · 2026-10-15
- Claude Code error mentions 'Fable 5.1' and 'Opus 5', hinting at unreleased models — adonis_singh · 2026-10-15