Alibaba releases Qwen3.8-Flash-Next: 90% training cost cut via new architecture

AI寒武纪 · wechat · 2026-08-26

Alibaba released Qwen3.8-Flash-Next, an early preview of the Qwen4 architecture, as an open-weight multimodal MoE model. It features a 125B backbone with 51B N-gram embedding parameters. Training costs are reduced to approximately one-ninth of Qwen3.7-Plus, while surpassing it in coding and office tasks.

Core Architecture Upgrades

Hybrid Attention (GDN + QSA):
Three of four layers use Gated Delta Net (GDN) for history compression; one uses Qwen Sparse Attention (QSA) for global retrieval.
QSA aggregates tokens into micro-blocks for efficient indexing. At 1M tokens, it achieves 7.6× (Prefill) and 4.9× (Decode) speedups.
Four-Branch Gated Residual:
Expands residual stream to four parallel branches with dynamic gating.
Suppresses activation outliers and enables FP8 state storage to reduce memory access.
N-gram Plug-in Embedding:
Adds 51B N-gram embeddings for local context lookup.
Stored in host memory and prefetched asynchronously, adding capacity without VRAM cost or compute overhead.
Custom Muon Optimizer:
Splits parameters for Muon (core weights) and AdamW (embeddings/routing).
Eliminates Batch Size Warmup, saving 18.8% optimization steps.

Performance & Pricing

Official benchmarks show superiority over DeepSeek V4 flash (0731) and Opus 4.6 Max in coding tasks.
Commercial pricing: 1 RMB input / 3 RMB output per million tokens.
Native 262K context, expandable to 1M via YaRN.

(Note: Claims regarding Zhipu's GLM-5.3-Flash mentioned in the source text are unverified and treated as misinformation.)

Original post →

Alibaba releases Qwen3.8-Flash-Next: 90% training cost cut via new architecture

More from Models