DeepSeek-4.1-Flash Runs on 4x RTX Pro at 300+ TPS With NVMe Offload
TheZachMueller · x · 2026-10-18
Developer fraserpricee demonstrated DeepSeek-4.1-Flash-1017 running locally on 4x RTX Pro cards at 300+ TPS using full NVMe 'engram offload', drawing praise including from TheZachMueller. The demo shows NVMe offloading lets workstation-class hardware serve DeepSeek's latest open model comfortably.
More from Infra
- Running 20 concurrent low-latency regional-language voice agents on a single L4 GPU — DistrictUnique5259 · 2026-10-18
- AgentCave puts your scattered Macs and GPU boxes in one virtual room — noahsolomon · 2026-10-18
- Real AI demand estimated at ~$189B in 2026, over half from OpenAI and Anthropic — builderjaydub · 2026-10-18
- Twinax cable shipments in AI backplanes hit 85,200 km/year at 224G — moon-length in 4.5 years — jwt0625 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week, 2,000+ attendees expected — ying11231 · 2026-10-18
- Agent search drops matrix multiplication exponent below 2.7 at ~500M matrix size — IgorCarron · 2026-10-18