vLLM MTP3 validation: AIME25 up to 0.9667 with prefill latency cuts
vllm_project · x · 2026-10-15
vLLM validated MTP3 against the baseline on B300 DEP8:
- AIME25: 0.9500 → 0.9667
- GPQA: 0.9268 → 0.9242
- OCRBench: 0.896 → 0.895
- Two validation rounds completed with zero request errors
Companion sparse-prefill optimizations: indexer logits kernel latency down 43%, top-k kernel latency down 84%, overall prefill latency down 2.2% (measured with MTP3, FP8 KV, 60K-token input, MegaMoE path). The work also removes 7 runtime JIT warmups and caps chunks at 4,096 tokens.
The team notes these are results from the tested build only — not a claim that final-release validation is complete or that model quality broadly improved.
Related event: vLLM Shares P/D Prototype and MTP3 Benchmark Results(2 posts)→
More from Infra
- First PyTorch Madrid meetup demos llm-d: K8s-native distributed LLM inference on vLLM — SergioPaniego · 2026-10-15
- Tsinghua's Latent Compaction Compresses KV Cache 32× While Keeping 96.7% Retrieval Accuracy — Yingfa Chen · 2026-10-15
- Starlink Wi-Fi spotted on Austrian Airlines, speedtest shows 157 Mbps down at 43ms ping — mitsuhiko · 2026-10-15
- Banks try to offload AI data center debt they just underwrote — jazir55 · 2026-10-15
- ContinuityBench: Stateful Failover Preserves LLM Conversations at 99.2% Across Providers — its_vayishu · 2026-10-15
- When Claude refuses your insane GPU optimization, DeepSeek gets the call — mike64_t · 2026-10-15