vLLM adaptive verification now works head-free, using draft logits with online logistic calibration
vllm_project · x · 2026-10-15
vLLM announces a major simplification to adaptive verification: it previously required a drafter with a trained confidence head, but now works with draft models you already serve—MTP, EAGLE, DFlash and DSpark checkpoints without a head, no retraining.
Key insight: the signal was already in the draft logits. The drafter's own top-token probability ranks drafts as well as a trained confidence head, but overestimates acceptance rates; a tiny logistic fit trained online from the target's accept/reject decisions corrects the calibration.
In benchmarks, a single adaptive verification config at max draft length stays on the throughput–interactivity Pareto frontier at most concurrencies with no per-load tuning. Available now on main behind enableadaptiveverification, for draft models whose attention backends support variable-length verification. Built by Giancarlo Delfin.
More from Infra
- China May Not Even Hit 10 GW Total Accelerator Output by 2028, Argues Analyst — teortaxesTex · 2026-10-15
- China Now Builds ~0.5 GW of Accelerators Monthly, Bottleneck Shifting to Cluster MTBF — teortaxesTex · 2026-10-15
- Benchmarking coding agent harnesses and inference: lessons from $100Ks/month in research spend — abhijithneil · 2026-10-15
- Geometric self-attention: 14,000x smaller state than KV cache with better CE at 2K context — iskander · 2026-10-15
- Micron has returned 1,230% in three years, beating Nvidia, AMD and Broadcom 3-4x — Beth_Kindig · 2026-10-15
- vLLM + NVIDIA wideEP Co-Design Makes GB300 NVL72 7.2x Better Perf per TCO — woosuk_k · 2026-10-15