Mistral Large 4 'Le Chonk' lands on Hugging Face: 1T-param MoE, 49B active, 1M context
ariG23498 · x · 2026-10-29
Mistral's flagship Mistral Large 4 (aka Le Chonk) is on Hugging Face, with a detailed explainer post:
- Multimodal (text+image in, text out), MoE: 1T total params, 49B active, 1M-token context
- Official FP8 and NVFP4 variants, plus GGUFs from Unsloth and the llama.cpp team
- Released under the Mistral Ecosystem Open-Weight License (an MIT derivative restricting model-as-a-service)
- Architecture combines a large expert pool, sparse attention, and a Pixtral vision encoder: per-token expert selection plus an attention indexer choosing earlier token positions — two distinct forms of sparsity
- Especially strong at cybersecurity, coding agents, and multimodality per Mistral
The post also covers inference support in Transformers, vLLM, llama.cpp, and fine-tuning with TRL.
More from Models
- Mistral Open-Sources Trillion-Parameter Coding Model Le Chonk, Still Trails DeepSeek and GLM — The Decoder · 2026-10-29
- Mamba co-author credits SmolLM3's open recipe and checkpoints for enabling research — srush_nlp · 2026-10-29
- ThursdAI Recap: Gemini 4 Carbon, Genie 4, Anthropic IPO Rumors and More — WolframRvnwlf · 2026-10-29
- Collecting prompts that reliably make GPT answer incorrectly — Qaukasus · 2026-10-29
- Independent eval finds DeepSeek V4.1 Flash-1017 near-parity with closed models on safety — FangcunAI · 2026-10-29
- User wants Dots removed from ChatGPT sub to save 25% — johnlindquist · 2026-10-29