DeepSeek v4 runs at 47 tok/s on single GPU, passes 370k needle test
EAccelerate_42 · x · 2026-08-21
DeepSeek v4 Flash 0731 has been optimized to run smoothly on a single DGX Spark unit. Using EXL3 quantization, it achieves 47 tok/s generation speed and 1024 tok/s prefill, while passing a 370k token needle test. The performance is enabled by native NVFP4 KV cache, with quality comparable to Q4KM / Q5 GGUF.
Related event: DeepSeek-V3-Flash Runs at 47 tok/s on a Single $47 Setup(2 posts)→
More from Infra
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16
- vLLM ships Day-0 LoRA RL for Muse Spark Open, lifting AIME from 0.56 to 0.85 — vllm_project · 2026-10-16
- DSpark draft model for Muse Spark Open proposes 7 tokens per pass with 23-42% acceptance — vllm_project · 2026-10-16
- Cloudflare Stream's WebRTC ultra-low-latency live streaming is now Generally Available — ritakozlov · 2026-10-16
- YC-backed Overshoot launches inference optimization, claims $5M saved for AI teams — ycombinator · 2026-10-16