Running DeepSeek V4.1 Flash locally at 15 tok/s on an RTX 3090 build for ~$2.5k
casper_hansen_ · x · 2026-11-08
Developer casperhansen shows DeepSeek V4.1 Flash 1017 running locally on a self-built RTX 3090 rig at 15 tokens/s, calling it fully viable for about $2.5k in hardware. His roadmap: build his own inference engine, an undetected browsing plugin, a new quantization framework, then iterate. Benchmark screenshots show the NVMe mode decoding at 17.5–20.2 tok/s across 1k–32k prompts with 99.6% top-1 agreement vs exact decode; a 120k-token prompt prefills at 657 tok/s and decodes at 18.5 tok/s, and warm prefill cuts time-to-first-token on long contexts from 33s to 8s.
More from Infra
- Hyperbolic CEO on compute market-making: bridging 3-year contracts and 3-week needs — YiMaTweets · 2026-11-08
- Ben Bajarin mocks 'we don't need data centers' take: CSPs were 'useless' too — BenBajarin · 2026-11-08
- KV Cache Quantization Tested Across 35 Quants: 8-bit Is Essentially Free for Qwen3.8 27B — Hefty_Wolverine_553 · 2026-11-08
- Developer burns $4,600 on rented GPUs with zero training steps, finishes same RL run on Tinker for under $100 — simonguozirui · 2026-11-08
- If compute moves to space, will data center critics still protest progress itself? — Dr_Singularity · 2026-11-08
- Open-source Strata runs DeepSeek V4.1 on an RTX 3060 at 9-11 tokens/sec — alex_verem · 2026-11-08