4x Faster Kimi-K3 Decoding: Red Hat Releases DSpark Speculator
woosuk_k · x · 2026-08-14
Red Hat AI has released DSpark, a speculative decoding model optimized for Kimi-K3, delivering massive inference speedups.
- Single-stream interactivity: Boosts speed from 110 to 435 tok/s/user on math reasoning.
- Throughput: Delivers 3.5x the output throughput at matched interactivity under load.
- Long-context stability: Using a 2048-token sliding window attention across all 5 draft layers, the model maintains steady acceptance rates up to 20K context across 10 LongBench domains.
- Built on the vLLM project, it is particularly performant for long-context agentic workloads.
Related event: Red Hat Unveils DSpark Decoder, Greatly Boosting Kimi-K3 Inference(2 posts)→
More from Infra
- MLCommons launches MLPerf Endpoints, a benchmark that measures inference endpoints the way buyers actually purchase them — TheKanter · 2026-10-16
- Neocloud M&A decoded: CoreWeave, Nebius, Crusoe, Nscale all bet margins vanish beyond GPU scarcity — AccBalanced · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16