Mistral Large 4: 1T-param MoE shows 16x KV-cache cut hurts agentic skills
_AndrewZhao · x · 2026-11-03
Mistral published its technical report for Mistral Large 4, a 52B-active / 1T-total multimodal MoE model built for reasoning and long-horizon agentic tasks, released exclusively on alphaXiv.
Key finding: aggressively compressing KV-cache (initially 16x) hurts long-context agentic performance even when language modeling metrics look unchanged. The fix: restore full attention to more layers, then adopt DeepSeek Sparse Attention to select the 2,048 most relevant tokens instead of attending to everything.
- Scores 41% on Terminal-Bench 4.0, 19.7% on FrontierCode 2.0, 50.9% on the Artificial Analysis Cyber Index
- Serving stack combines sparse attention, optimized context parallelism, prefill-decode disaggregation, and speculative decoding
- Calibrated NVFP4 quantization shrinks the checkpoint to 606 GB while matching FP8, fitting on four GB300 GPUs
- Multi-teacher distillation offers high/medium/low reasoning-effort modes
Related event: Mistral Large 4 Tech Report: 1T-Param MoE Trained on Just 10MW(9 posts)→
More from Infra
- KVCache Meetup Korea dives into LLM serving: CXL pooling, roofline analysis, batch scheduling — PyTorch · 2026-11-03
- FuriosaAI's new round values it at 662x last year's sales, no chipmaker joined — YvesMulkers · 2026-11-03
- Just upgrading onnxruntime-gpu 1.23→1.31 with CUDA 13 made SAM 3 inference 3.8x faster — MaziyarPanahi · 2026-11-03
- Dev argues buying GPUs for local AI is just an expensive hobby — API and rented GPUs are always cheaper — Angaisb_ · 2026-11-03
- Local inference devs grapple with models breaking their own CoT on streamed tokens — SilentMobius · 2026-11-03
- FLAMIA: MXFP8 microscaling attention kernel for fast, causally-safe training on Blackwell — yresearch · 2026-11-03