vLLM's Speculators v0.8.0 ships unified CLI, PyPI Mooncake connectors, fused Triton loss kernel
vllm_project · x · 2026-09-08
Red Hat AI released Speculators v0.8.0 for training speculative decoders, building on the DSpark/DFlash foundation:
- Unified speculators CLI covering regeneration, preprocessing, and online/offline training
- Mooncake hidden-state backend as a standalone hs-connectors package on PyPI, enabling multi-node runs
- Single fused Triton online-softmax loss kernel for lower-memory training
- Experimental DFlash2 support and Laguna warm-start with DFlash weight remapping
- Reproducibility artifacts for eval runs
More from Infra
- Dot-com era left 95-98% of US long-haul fiber dark; AI is now lighting it up — jwt0625 · 2026-09-09
- Magic claims 50x pretraining efficiency: matches DeepSeek V4 Pro for ~$0.5M — magicailabs · 2026-09-09
- Thomson Reuters' frontier-competitive legal model Thomson trained for just $450K with curated data — schwarzjn_ · 2026-09-09
- Qualcomm confirms AWS deal is baked into its $15B FY29 data center revenue target — BenBajarin · 2026-09-08
- Data center construction spend jumps $25B in six months, job openings top 300k — AccBalanced · 2026-09-08
- CDOs backed by GPU leases likely coming as compute financialization accelerates — AccBalanced · 2026-09-08