763B DeepSeek-V4.1-Flash runs locally at 500+ tok/s on 4x RTX Pro 6000
TheZachMueller · x · 2026-10-19
Developer fraserpricee shared a Megakernel local inference milestone: running DeepSeek-V4.1-Flash-1017 (763B params) locally at full precision on 4x RTX Pro 6000 Blackwell Max-Q, hitting 502.7 tok/s output with a standard coding prompt, plus 10,943 tok/s prefill and 244ms TTFT.
Technical details: vLLM engine, FP8, 5 DSpark tokens, full SSD ngram offload, 524,288 token context. Still work-in-progress; a full recipe is promised to follow.
More from Infra
- llama.cpp changes default server port from 8080 to 9931 in v0.7.0 — 413205 · 2026-10-19
- Microsoft pulls out of £2B UK AI data center as grid delays push power into 2030s — Polymarket · 2026-10-19
- Local Qwen3 27B generates a full video in ~2 hours, completely free — aziz4ai · 2026-10-19
- Poliscope Made Germany's 4,800 Council Portals Searchable on One Qdrant Server, Cutting Latency from 20s to 0.87s — qdrant_engine · 2026-10-19
- GPUs literally falling off the bus: MCIO riser triggers NVIDIA Xid 79 errors — TheZachMueller · 2026-10-19
- selflift-Fish: 20s 1080p MiniMax H3 video on a 16GB GPU, 2.2x faster — Hot_Store_5699 · 2026-10-19