Inferact’s Kimi-K3-DSpark draft model reuses MLA caches to speed up vLLM serving

vllm_project · x · 2026-07-27

• Inferact’s Kimi-K3-DSpark is an MLA-native draft model for accelerating Kimi K3 inference on vLLM.
• The draft is trained on target hidden states extracted from vLLM itself, so it learns from the exact distribution it will later serve.
• DSpark combines a block-diffusion backbone (5 dense layers drafting 7 tokens in parallel), a rank-256 sequential Markov head, and a confidence head.
• Because the draft mirrors Kimi K3’s MLA attention, draft and target share a single compact KV layout, enabling reuse of the same cache management and PD-disaggregated serving path.
• The card also notes that acceptance is higher on deterministic text like code and lower on high-entropy generation like creative writing, using production-like settings such as tensor-parallel-size=8, temperature=1.0, and topp=0.95.

Original post →

More from Infra

• Claude chat indexing incident sparks a push for confidential-compute AI — bittingthembits · 2026-07-27
• Moonshot open-sources MoonEP as open models vs closed labs debate intensifies — KyeGomezB · 2026-07-27
• Kimi K3’s 2.5x scaling-law gain draws praise for training efficiency — andrew_n_carr · 2026-07-27
• Kimi K3 goes live on Nebius with 1M-token context and a 57 AA score — teortaxesTex · 2026-07-27
• AI agent finds a longstanding Bun Node-compat bug in `child_process.spawn` — steipete · 2026-07-27
• llama.cpp adds support for Nanbeige4.2 in pull request 25994 — pmttyji · 2026-07-27

Infra channel →