Mistral Large 4 tech report: 1T-param MoE, NVFP4-quantized to 606GB, runs on 4 GB300 GPUs
_AndrewZhao · x · 2026-11-04
Mistral published the story of building Le Chonk (Mistral Large 4) as an alphaXiv-exclusive tech report. Key facts:
- Architecture: a 52B-active, 1T-total-parameter multimodal mixture-of-experts model targeting reasoning and long-horizon agentic tasks
- Data pipeline: a scalable agent-based pipeline that authors and verifies executable environments for software engineering, cybersecurity, and knowledge work
- Benchmarks: 19.7% on FrontierCode 2.0, 41% on Terminal-Bench 4.0, 50.9% on the Artificial Analysis Cyber Index
- Serving: sparse attention, optimized context parallelism, prefill-decode disaggregation, and speculative decoding; calibrated NVFP4 quantization shrinks the checkpoint to 606GB while matching the FP8 reference, enabling deployment on four GB300 GPUs
- Multiple-teacher distillation offers high/medium/low reasoning-effort modes to trade token consumption against performance
More from Models
- Abliterated GLM-5.3 pulls off red-team exploit in 3:41 for $0.02, stock model refuses — bratton · 2026-11-04
- Tencent Hunyuan unveils Hy4, open weights coming soon — heatOverflower · 2026-11-04
- OpenAI's next model won't be cheap, quips likening it to an 800B active-parameter frontier — teortaxesTex · 2026-11-04
- Gemini refuses to process a trivial screenshot containing only a number — Regular_Elk4015 · 2026-11-04
- LMArena opens Agent Arena for crowdsourced AI agent evaluations — arena · 2026-11-04
- Tencent releases full benchmark results for Hunyuan 4 — EmetInteractive · 2026-11-04