Qwen3.8-Flash-Next architecture: 125B params, sparse MoE, 262K context
Alibaba_Qwen · x · 2026-08-26
vLLM announces support for Qwen3.8-Flash-Next, an ultra-sparse MoE multimodal model. It features 125B total parameters (including a 51B N-gram table) with only 6B active per token and native 262K context support (extendable to 1M via YaRN). The architecture combines Gated DeltaNet for history compression, Qwen Sparse Attention for precise retrieval, and MTP for speculative decoding. The N-gram table can be offloaded to host memory.
More from Infra
- SK Hynix ties new US plant to looser chip equipment exports to China, sources say — pstAsiatech · 2026-10-15
- Perplexity's new embedding models: index with a 9B model, search with 0.6B — antoine_chaffin · 2026-10-15
- Unsloth ships GGUF quants for Nemotron 3.5: Bolt-74B runs in 24GB RAM — danielhanchen · 2026-10-15
- vLLM-Omni ships Day-0 serving for FLUX 3 Image Klein with BF16, FP8, NVFP4 support — vllm_project · 2026-10-15
- NVIDIA's Nemotron 3.5 Super VL lands training and inference support on Applied Compute — ypatil125 · 2026-10-15
- CoreWeave bond yield tops 16% as loans outlast customer contracts by ~2 years — tengyanAI · 2026-10-15