DeepSeek V4.1 Flash on 4x DGX Spark: weights alone are 511GB, the 494 tok/s claim doesn't fit

Aroochacha · reddit · 2026-10-16

Reddit user Aroochacha independently reproduced Patrick Moorhead's 4x DGX Spark setup for DeepSeek-V4.1-Flash (reported at 494 tok/s peak via Wccftech) and found serious omissions and misreporting.

Problems with the coverage

• Neither Moorhead's tweet nor the article specified the serving engine, recipe, settings, or test prompts; the tweet's first image shows only 69 t/s
• Wccftech's "Memory footprint: 476 GB" is actually on-disk size in GiB, not runtime memory. Moorhead later credited an NVIDIA employee for the setup, saying he "may/may not have funded it"

Reproduction findings

• Weights alone are 511 GB, but only 80–92% of the 512 GB pooled memory (410–470 GB) is usable; allocations above 0.92 (117 GB per Spark) crash or hang, and memory fluctuates during runs
• Engram tables (196 GB) must be streamed from each node's local NVMe with a 4 GB row cache, or the model won't run at all

Gotchas and fixes

• Startup OOM (ibvregmr Cannot allocate memory / CUDA OOM): page cache from reading weights fills shared memory; drop caches during startup and enable SGLang's post-load cache drop
• NCCL init failures in the first minute are a startup race — just run serve again
• Working setup: 4x DGX Spark on a 100 GbE switch, SGLang TP4 with knapcio's recipe, DSpark speculative decoding, 262K context, temperature 0

Original post →

More from Infra

• Amazon Proposes Per-Request Validated Evidence Caching for Production RAG — _reachsumit · 2026-10-16
• LinkedIn's TransX report: sequence transduction for recsys, +6.02% CTR in A/B tests — _reachsumit · 2026-10-16
• llama.cpp PR boosts CPU inference speed with merged AVX2 tiled accumulators — jacek2023 · 2026-10-16
• Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
• Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
• One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16

Infra channel →