DeepSeek v4.1 hits 340 tok/s single-stream locally; 700+ tok/s claims called benchmark tuning
TheZachMueller · x · 2026-10-28
ZenMagnets modified @fraserpricee's inference recipe and hit 340 tok/s single-stream (1017 tokens) serving DeepSeek v4.1 on tp4 RTX 6000 NVFP4 — the fastest he's seen on his own hardware, using a BRRR/B12X hybrid that switches to native B12X above 8 scheduled tokens. He argues earlier 160 tok/s results stemmed from an incomplete recipe, and that the claimed 700+ tok/s is largely "racetrack optimization": on standardized llm-inference-bench, Fraser's released recipe gets 160 tok/s at C1 and 400 at C8. The attached benchmark table shows combined decode throughput across C1–C32 and 0k–128k contexts (peak 1755 tok/s at C32/0k), with 29/30 valid cells and clean quality tests (LAVD 30, Estonia 30 all passing). Recipe released in-thread.
More from Infra
- Data centers become the new front line as AI, cyber, physical and OT threats converge — ChuckDBrooks · 2026-10-28
- preowned turns macOS built-in Apple models into an OpenAI-compatible local API — pcuenq · 2026-10-28
- 2,100+ tok/s/user for DeepSeek V4.1 Flash on 8× AMD MI355X, 2.8x faster than open-source — JiaZhihao · 2026-10-28
- LFM3-VL-2B Runs Real-Time Video Captioning In-Browser via WebGPU — xenovatech · 2026-10-28
- Half of Agent Infra users are no longer human: Cloudflare, Vercel and Supabase numbers — FinanceYF5 · 2026-10-28
- Phonon on-device TTS hits 80K downloads with PyTorch-free Rust runtime — mattturck · 2026-10-28