Hybrid-Spark switches drafters per decoding step, up to 1.62x faster than DSpark on code edits
Julianfmack · x · 2026-11-10
Hybrid-Spark proposes mixing speculative-decoding drafters instead of relying on one drafter throughout: it switches per request and per step between a model-based drafter (DSpark) and a model-free n-gram drafter.
Benchmarks show up to 1.62x throughput gains over DSpark alone on code-edit workloads; charts compare batch sizes 4 and 128 across edit-distance buckets, with the hybrid approach consistently on top.
More from Infra
- Running open-weight models locally: Kimi K3.1 needs 594GB, ~$38k for 15 tok/s — ravikantagrawal · 2026-11-10
- InfraRSI Applies Recursive Self-Improvement to Training Infra: 2.75x Speedup, MFU Lifted from 17.4% to 30% — TokenRhythm · 2026-11-10
- ClickBench Playground Lets You Run SQL Against 110+ Databases Side by Side — mgill25 · 2026-11-10
- Musk: new Grok training stack derived from Starlink's in-house engineering — elonmusk · 2026-11-10
- 4x RTX 3090 vs one RTX 5090: Malaysian builder weighs VRAM against newer architecture — whity2773 · 2026-11-10
- Musk says the Grok training stack is mostly C with a little C++ — elonmusk · 2026-11-10