Leaked Mistral Large 4 abstract: 1T-param MoE with 52B active, fits on 4 GB300 GPUs

qtnx_ · x · 2026-11-03

A document dubbed the "Le Chonk report" surfaced with a paper abstract for Mistral Large 4 (ML4)—unverified by Mistral.
Described as a 52B-active, 1T-total-parameter multimodal MoE built for reasoning and long-horizon agentic tasks; the team quantified KV-cache compression's penalty on agentic benchmarks and addressed it via model surgery plus sparse-attention adaptation.
Trained with a scalable agent-based pipeline that authors and verifies environments for software engineering, cybersecurity, and knowledge work.
Reported scores: 19.7% on FrontierCode 2.0, 41% on Terminal-Bench 4.0, 50.9% on the Artificial Analysis Cyber Index.
Serving combines sparse attention, context parallelism, prefill-decode disaggregation, and speculative decoding; calibrated NVFP4 quantization shrinks the checkpoint to 606GB while matching FP8 evals, enabling deployment on four GB300 GPUs. Multi-teacher distillation offers high/medium/low reasoning-effort modes.

More from Models