Red Hat's DSpark Speculator Boosts Kimi-K3 Throughput by 3.5x
teortaxesTex · x · 2026-08-14
Red Hat AI released the DSpark speculator model for Moonshot's Kimi-K3, significantly enhancing inference performance.
- Performance: Single-stream interactivity jumps from 110 to 435 tok/s/user on math reasoning. Delivers 3.5x output throughput at matched interactivity under load.
- Long Context: Thanks to sliding window attention (2048-token window across all 5 draft layers), the model maintains steady acceptance out to 20K context across 10 LongBench domains.
- Training: Since Kimi-K3 does not fit on a single GB300 node, training used a multi-node setup, streaming hidden states between disjoint node groups via a Mooncake store.
Related event: Red Hat Unveils DSpark Decoder, Greatly Boosting Kimi-K3 Inference(2 posts)→
More from Infra
- Dev publicly asks neoclouds for discounts on a $30k GPU compute purchase — Sauers_ · 2026-10-16
- New hardware day: local LLM deployment the moment it arrives — lxfater · 2026-10-16
- Together AI's score centering fixes RL training collapse under quantized inference — nrehiew_ · 2026-10-16
- Inference is splitting prefill and decode across specialized chips, not just H100s — AccBalanced · 2026-10-16
- Rebasely launches in beta: Git hosting built for humans and coding agents — handotdev · 2026-10-16
- Drama3 TTS lands in audio.cpp: up to 5x real-time on RTX 5090, <8GB VRAM mode too — Acceptable-Cycle4645 · 2026-10-16