TensorSharp Benchmark: Speculative Decoding Doubles DeepSeek Speed
fuzhongkai · reddit · 2026-08-03
The open-source local inference engine TensorSharp now supports DSpark speculative decoding, releasing benchmark results for the DeepSeek-V4-Flash-0731 model.
On a setup with 4x Nvidia A40 GPUs, enabling DSpark significantly boosted inference speeds:
- Short generation: 25.6 to 44.5 tokens/s (1.74x)
- Long generation: 26.4 to 40.3 tokens/s (1.53x)
- 10K-token document processing: 25.3 to 51.3 tokens/s (2.03x)
TensorSharp is a native open-source LLM inference engine supporting CUDA, Vulkan, Metal, continuous batching, and multimodal capabilities.
Related event: TensorSharp Engine Doubles DeepSeek Inference Speed(4 posts)→
More from Infra
- Valtz v0.1.0.11 adds FLUX.3 support, local image gen up to 4K on Mac — TgoAI · 2026-10-16
- Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training — BaiduResearch · 2026-10-16
- GYRO quant of Qwen3.8-Flash-Next tested: near Q3 quality at much smaller size — PlasticRevenue4601 · 2026-10-16
- RL train-sample systems are frontier AI's hardest infra problem — and a trade secret — brianryhuang · 2026-10-16
- Local tuning of GLM-5.3-Flash on 2×GB10 lifts decode from 49 to 58 tok/s — EAccelerate_42 · 2026-10-16
- 3x DGX Spark cluster runs GLM-5.3-Flash locally behind a proxy, 1.75M tokens/day at 119W — Teknium · 2026-10-16