Red Hat Releases New DSpark Models, Boosting vLLM Inference Speed by 4x
vllm_project · x · 2026-08-08
Red Hat AI has released three new DSpark speculators on Hugging Face to accelerate LLM inference through parallel drafting. The models are designed for Qwen3.6-35B-A3B, gemma-4-31B, and GLM-5.2.
DSpark drafts a whole block of tokens in a single pass and then verifies them losslessly. On the GLM-5.2 model, this technique achieves a 4.0x speedup in decoding, boosting single-request generation from 137 to 551 tokens/sec. It also delivers a 1.57x higher peak throughput (from 1,402 to 2,203 tokens/sec) with an acceptance length of 5.47 on math reasoning tasks. These models can be served using @vllmproject.
More from Infra
- Optuna creator Akiba: when 2 hyperparameters cost $20M, what's left for BO/HPO — Stefania_druga · 2026-10-16
- Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
- LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16
- llama.cpp PR boosts CPU inference speed with merged AVX2 tiled accumulators — jacek2023 · 2026-10-16
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16