Maxwell-V1-Flash: 35B MoE with 3B active params, 256K context, 6.7x faster attention

pmttyji · reddit · 2026-11-17

Maxwell Team released Maxwell-V1.0-Flash on Hugging Face: a MoE LLM with 35B total parameters (3B active per token, excluding embeddings) and 256K context.

Key techniques

Training: 26T multilingual tokens, Muon optimizer on most hidden weights; dense start, block-sparse attention introduced at 128K mid-training, staged extension to 256K. Script-aware tokenizer stays compact for low-resource Southeast Asian languages (Thai, Lao, Myanmar, Khmer, Shan).

Post-training: two-stage all-domain SFT, then separate RL branches for general, verifiable reasoning (math/science/logic/code), and agentic tasks (search, terminal, function calling, SWE), merged via multi-teacher on-policy distillation.

Original post →

More from Models

Models channel →