Lawyer runs DeepSeek Flash at near-NVFP4 quality on eight 2018 V100s: 524K context, 100+ tok/s, fully local
TumbleweedNew6515 · reddit · 2026-11-13
A lawyer known for hoarding GPUs published his fifth local-deployment build: DeepSeek Flash 1017 running on eight V100-SXM2-32GB cards at near-NVFP4 quality, with 524K context, 118.9 tok/s single-request decode, and 159 tok/s with two concurrent chats — fully local, powering his legal workflow.
The Volta Max uPLIft approach
- Keep a 2-bit record of every routed expert on the cards, then upgrade the most-used experts (measured on his legal harness traffic plus three public prompt sets) to NVFP4 weights one by one until VRAM is full
- 5,949 of 15,360 routed experts sit at exact NVFP4 precision; 78-85% of decode routes land on them
- 95.1% next-token match vs NVIDIA's full NVFP4 pack (2-bit build: 92.2%), KL divergence 0.079 vs 0.196 nats, multiple-choice flips 1.6% vs 15.4%
- Every routed expert fits in VRAM during decode; nothing crosses PCIe
He also runs two DGX Sparks handling 3-4 concurrent DeepSeek agent sessions at 1M context, plus 8 more V100s and a 512GB ECC RAM server for expansion. Notably, newer models (Opus 5.5, Fable 5.2) now handle the configuration, custom Volta kernels, and activation-guided expert requantization automatically — the first time his frontier-built legal harness has been fully replaced by a local stack. Code and benchmarks are open-sourced on GitHub.
More from coding & agent
- After a wave of breaches, Huntley says the answer to AI is engineering, not more AI — rseroter · 2026-11-13
- isometric-cities: one prompt turns real open map data into isometric 3D cities via agents — zsakib_ · 2026-11-13
- JoMoHa jointly RL-trains model and harness, hits 97.0% on ALFWorld — omarsar0 · 2026-11-13
- YC F26 team open-sources runtime to run 10,000 agents in one session — KlausCodes · 2026-11-13
- Open-Source Pipeline Turns LiDAR + OpenStreetMap into Isometric Pixel-Art Cities with FLUX.2 — zsakib_ · 2026-11-13
- X, MIT, UW & Stanford Launch WSDM Cup for AI Agents Writing Community Notes, $10k Prizes — NathanpmYoung · 2026-11-13