vLLM's MTP3 validation: AIME25 up to 0.9667, prefill kernels up to 84% lower latency
vllm_project · x · 2026-10-15
vLLM validated MTP3 against baseline on B300 DEP8: AIME25 improved from 0.9500 to 0.9667, while GPQA and OCRBench stayed flat, with zero request errors across two rounds. The team stressed these are tested-build results, not final-release validation, and small deltas don't establish general model-quality gains. Alongside, sparse prefill optimizations cut indexer logits kernel latency 43%, top-k kernel 84%, and overall prefill 2.2%, removing 7 JIT warmups and capping chunks at 4,096 tokens.
More from Infra
- Autonomous sells personal AI computers from $26,100 that keep data in the room — dee_hw · 2026-10-15
- vLLM's bitwise-consistent RL: zero KL on Nemotron NVFP4, 91.3% throughput on DeepSeek-V4-Flash — DynamicWebPaige · 2026-10-15
- vLLM achieves bitwise-consistent RL training: DeepSeek-V4-Flash hits 0 diff over 100 steps — vllm_project · 2026-10-15
- vLLM Achieves Bitwise-Consistent RL Training, Fixing KL Blowups and Score Collapse — vllm_project · 2026-10-15
- Analyst Ben Bajarin: The AI Industry Needs a Lot More Compute — BenBajarin · 2026-10-15
- Next-gen GraalVM benchmarks: 4x faster startup, 2x less memory, no build step needed — RealGeneKim · 2026-10-15