Mistral Large 4 open-weights: 1T-param MoE, leads open models in coding
RobertTLange · x · 2026-11-03
Mistral released its flagship Mistral Large 4 (Le Chonk) with open weights. Robert Lange's technical rundown:
- Architecture: 1T-param MoE (52B active), natively multimodal, 1M context; 72 layers of MLA + DeepSeek-style sparse attention where an indexer picks 2,048 keys/query
- Mid-pretraining surgery: SWA+CLA hybrid hurt SWE-bench at equal perplexity → reverted to full attention, YOCO kept; learned RoPE freqs + SSMax; NoPE every 4th layer; 70 KiB KV/token
- Experts: 128+1 experts, top-4, softmax router; built-in 3-layer DSpark drafter
- Benchmarks: leads open peers on FrontierCode 2.0 (19.7%) and Terminal-Bench 4 (41%)
- Deployment: NVFP4 build is 606 GB, serves on 4× GB300
The official blog positions the model for agentic coding and defensive cybersecurity, with three reasoning modes for cost-performance balance and a sovereign-AI, self-hostable European pitch.
Related event: Mistral Releases Large 4 Technical Report with Open Weights(13 posts)→
More from Models
- JFPuget's team takes 3rd on both Kaggle ARC-AGI competitions ending the same day — JFPuget · 2026-11-04
- DeepMind-Stanford paper: a red heart in your prompt flips code from 0% to 100% recursive — rohanpaul_ai · 2026-11-04
- A Red Heart Can Steer Code: DeepMind-Stanford Paper Probes Visual Meaning in VLMs — rohanpaul_ai · 2026-11-04
- DeepSeek open-sources DeepInfer inference engine delivering 26,000+ tokens/sec per GPU — bycloud · 2026-11-04
- DeepMind RL-trained Gemini doubles correct psychiatric differential diagnoses in blind study — SRSchmidgall · 2026-11-04
- Particle-1 post-trained on DeepSeek V4.1 Flash cuts output tokens ~45% — heyneighbor · 2026-11-04