Z.ai Releases GLM-5.3-Flash: Hybrid Sparse+Linear Attention Architecture
No_Afternoon_4260 · reddit · 2026-08-26
Z.ai released GLM-5.3-Flash, the first natively multimodal model in the GLM-5 series and the first open-weight release of the glm5next architecture. Pitched as outperforming GLM-5.2 at one-tenth the price and approaching Claude Opus 4.8 on coding and agentic benchmarks.
Key Highlights:
- Hybrid Sparse + Linear Attention: 45 layers structured as repeating blocks of 3x KDA linear attention followed by 1x DeepSeek-style sparse attention (34 linear / 11 sparse layers). Sparse layers use a lightning indexer with a top-k budget of 2048 tokens to reduce long-context serving costs.
- Manifold-Constrained Hyper-Connections (mHC): Widened residual streams with manifold-constrained mixing to improve scaling efficiency.
- Natively Multimodal: A 24-layer ViT encoder supports both image and video tokens, trained on a 30T-token multimodal corpus.
- MTP Head: Weights include 1 next-N prediction layer for speculative decoding (5 tokens in official vLLM recipe).
- FP8 First: Primary repo is FP8 (e4m3), with a separate BF16 repo available.
Specs: 320B total params (18B active), MoE (288 routed, 8 active), 1M context window, MIT License.
Related event: Zhipu Open-Sources GLM-5.3-Flash: 320B MoE Multimodal at 1/10th the Cost(48 posts)→
More from Models
- China's StepFun releases Step 5 Preview weights: 600B MoE, 27B active, 1M context — Dr_Singularity · 2026-10-15
- StepFun open-sources Step 5 Preview: 600B MoE with 1M context, No. 2 on OpenRouter — Dr_Singularity · 2026-10-15
- Google says teams across the company are using Gemini 4 Argon on its hardest engineering problems — CurieuxExplorer · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- 797GB of n-gram counts in CPU RAM let 1B models match 57% more compute — PangWeiKoh · 2026-10-15
- Leak: Moonshot's Kimi flagship 'x4' done with pre-training, K3.1 in internal testing — lyraxana · 2026-10-15