DeepSeek Local Deployment: Troubleshooting Severe Speed Drop with Speculative Decoding
Easy_Werewolf7903 · reddit · 2026-08-09
A developer encountered severe performance degradation while locally deploying the DeepSeek-V4-Flash model using llama-server. On a rig with an RTX 4090 and RTX 6000 Pro (120GB VRAM total), using the MTP draft model for speculative decoding yielded a solid 30-40 tokens per second.
However, switching to the DSpark draft model caused generation speeds to plummet to 1-2 t/s. The user ruled out VRAM limitations and provided both complete launch configurations, asking the community for help in identifying the bottleneck in the DSpark parameters.
More from Infra
- MLCommons launches MLPerf Endpoints, a benchmark that measures inference endpoints the way buyers actually purchase them — TheKanter · 2026-10-16
- Neocloud M&A decoded: CoreWeave, Nebius, Crusoe, Nscale all bet margins vanish beyond GPU scarcity — AccBalanced · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16