Lawyer runs full DeepSeek-V4.1-Flash on 12x V100 at 157-184 tok/s, beating DGX Spark stacks
TumbleweedNew6515 · reddit · 2026-11-20
A lawyer upgraded his V100 server from 8 to 12 V100-SXM2-32GB cards (3 NVLink boards as 3 pipeline stages × 4-card tensor parallel, split at layers 14/28), fitting all 15,360 routed experts of DeepSeek-V4.1-Flash in NVIDIA's NVFP4 format entirely in VRAM — no compression, no system RAM, since 8 cards only hold 254 GiB vs. the 278.9 GiB expert layers.
Measured results (single stream):
- Code decoding at 157-184 tok/s from 1K-128K context, 153 tok/s at 256K; legal text 92-102, prose 83-95 tok/s
- 128K file cold read in 28s; first token under 1s once cached, 427 tok/s summed across 3 streams on one cached file
- Server configured for 524,288 tokens per request, 4 slots
He claims no published DGX Spark stack (3x or 4x) matches single-stream speed at any node count, at roughly half the price of 3 Sparks. The whole project is driven through Claude Code, which ran the benchmarks and wrote up the results. Repo, configs, logs and failure notes are all public.
More from coding & agent
- Kylon launches AI finance agents that paid $233K in supplier bills over four months — ying11231 · 2026-11-20
- TestSprite 4.0 launches as an independent test layer for AI-written code — Tegadesigns · 2026-11-20
- Dev builds video highlight feature with Gemini Neon, says Claude Opus 5.5 still wins on coding speed — vista8 · 2026-11-20
- VP of engineering: coding models report like bad direct reports — raw dumps, no TLDR — robleclerc · 2026-11-20
- GitHub open-sources Agent Host Protocol for cross-device agent sessions — rseroter · 2026-11-20
- 100% code-driven Blender squirrel film, made for $34.76 — no video gen models — dreamwieber · 2026-11-20