Alibaba releases Qwen3.8-Flash-Next: 90% training cost cut via new architecture
AI寒武纪 · wechat · 2026-08-26
Alibaba released Qwen3.8-Flash-Next, an early preview of the Qwen4 architecture, as an open-weight multimodal MoE model. It features a 125B backbone with 51B N-gram embedding parameters. Training costs are reduced to approximately one-ninth of Qwen3.7-Plus, while surpassing it in coding and office tasks.
Core Architecture Upgrades
- Hybrid Attention (GDN + QSA):
- Three of four layers use Gated Delta Net (GDN) for history compression; one uses Qwen Sparse Attention (QSA) for global retrieval.
- QSA aggregates tokens into micro-blocks for efficient indexing. At 1M tokens, it achieves 7.6× (Prefill) and 4.9× (Decode) speedups.
- Four-Branch Gated Residual:
- Expands residual stream to four parallel branches with dynamic gating.
- Suppresses activation outliers and enables FP8 state storage to reduce memory access.
- N-gram Plug-in Embedding:
- Adds 51B N-gram embeddings for local context lookup.
- Stored in host memory and prefetched asynchronously, adding capacity without VRAM cost or compute overhead.
- Custom Muon Optimizer:
- Splits parameters for Muon (core weights) and AdamW (embeddings/routing).
- Eliminates Batch Size Warmup, saving 18.8% optimization steps.
Performance & Pricing
- Official benchmarks show superiority over DeepSeek V4 flash (0731) and Opus 4.6 Max in coding tasks.
- Commercial pricing: 1 RMB input / 3 RMB output per million tokens.
- Native 262K context, expandable to 1M via YaRN.
(Note: Claims regarding Zhipu's GLM-5.3-Flash mentioned in the source text are unverified and treated as misinformation.)
More from Models
- GLM-5.3 unprompted reasons about how its actions affect its future training, worries Apollo — MariusHobbhahn · 2026-10-15
- Gemini 4 argon vs Opus 5.5 hands-on: 3 min for $3 vs 25 min for $8 — zacharynado · 2026-10-15
- SentinelOne says OpenAI did 4-6 weeks of reverse-engineering in 8 hours — OpenAI · 2026-10-15
- Qwen3.8 27B scores 23.57% on word-based addition without reasoning, 167/169 with it — dl_weekly · 2026-10-15
- Hacker turns EmbeddingGemma 2 into 288x realtime STT and a fast TTS — kastnerkyle · 2026-10-15
- Claude Fable 5.5 reportedly in partners' hands, release eyed next week — lyraxana · 2026-10-15