Mistral Large 4 quantized weights hit vLLM with ~2x throughput via built-in DSpark

vllm_project · x · 2026-10-29

The vLLM project released FP8 and NVFP4 (Blackwell) quantized weights for Mistral Large 4 (1T total, A52B MoE), with Mistral leading the implementation and support from vLLM contributors, Red Hat, inferact, and NVIDIA. Built-in DSpark delivers 1.92–2.16x output throughput in FP8 and 2.00–2.23x in NVFP4 versus the matching baseline (TP8, 200 real prompts, greedy decoding, ≤512 output tokens). Cross-layer KV sharing, context parallelism, and NIXL P/D serving handle long requests; a technical deep dive is linked.

Related event: vLLM Ships Built-in DSpark and Mistral Large 4 Quantized Weights with Up to 2.23x Throughput Gains(2 posts)→

Original post →

More from Infra

• UK's New Data Centre Covenant Aims to Fix Grid Queues but May Raise Build Costs — ersatzben · 2026-10-30
• BellowsKV shares one latent across layers, keeping 97-98% accuracy at 13-14x KV cache compression — burkov · 2026-10-30
• Google Cloud launches Rubin-powered A5X instances scaling to 80,000 GPUs per site — Beth_Kindig · 2026-10-30
• Sam Altman calls widespread local inference "quite terrible" for global DRAM allocation — Hesamation · 2026-10-30
• Microsoft's Maia 300: 2x chips per rack, 2.5x memory bandwidth, up to 7x throughput from 2027 — Beth_Kindig · 2026-10-30
• Evo 2 genomic model now runs on vLLM: 7B fits 1-megabase context on a single H100 — anshulkundaje · 2026-10-30

Infra channel →