Self-hosting DeepSeek V4.1 Flash: 160 tok/s single session, 30-50 with 5 parallel
gnukeith · x · 2026-11-08
gnukeith shares bandwidth math from self-hosting DeepSeek V4.1 Flash: about 160 tokens/s for a single session with high-quality near-lossless quantization, dropping to roughly 30-50 tokens/s per session across 5 concurrent sessions when optimizing for parallelism. He hopes for a breakthrough but admits physics may cap it.
More from Infra
- SpaceX plans 6 GW AI campus near Willcox, Arizona with dry cooling — aronchick · 2026-11-08
- François Fleuret asks: after a loss spike and recovery, is final loss worse, same, or better? — francoisfleuret · 2026-11-08
- ResearchBoost Agent speeds up real research workloads 2.51x on average across 12 tasks — YouJiacheng · 2026-11-08
- Dev builds Cray-2 emulator from scratch, runs a GPT-3-class LLM at 1 token per 4s — inductionheads · 2026-11-08
- Hyperbolic CEO on compute market-making: bridging 3-year contracts and 3-week needs — YiMaTweets · 2026-11-08
- Ben Bajarin mocks 'we don't need data centers' take: CSPs were 'useless' too — BenBajarin · 2026-11-08