Running full DeepSeek V4.1 Flash weights on 4x DGX Spark (~$28k): 1M context at 66 tok/s
ricklamers · x · 2026-10-26
ricklamers showcases a local "unrestricted" AI setup: four DGX Sparks ($28k all-in) running official, unquantized DeepSeek V4.1 Flash 1017 weights via switchless TP4. Key specs and benchmarks:
- 1M context, 16M total KV cache, 16 concurrent 1M sessions, 15 GB free per Spark after load
- Decode speeds (thinking on, non-greedy): 66 tok/s single stream, 96 at 2 streams, 173 at 16
- 100% needle pass across full depth
- Optional ABLIT=1 grafts a refusal-direction ablation overlay on attn.wob in layers 10-35 only; gated via HF token, first switch downloads 1.1 GB sidecar and rebuilds 80 GB rank cache; validated byte-for-byte with needle spot checks at 65k/262k tokens matching stock speeds
Suggested use case: home network cybersecurity.
More from Infra
- SpaceX files $40B GPU financing commitment with Apollo, already at 1.7GW — XFreeze · 2026-10-26
- Meta plans to raise up to $95B to fund massive AI expansion — Polymarket · 2026-10-26
- SpaceX lands $40B GPU finance lease with Apollo, the largest ever — firstadopter · 2026-10-26
- Nebius CEO: Debt Markets Closed for Neoclouds Last Week, Leaning on Customer Advances — zephyr_z9 · 2026-10-26
- CoreWeave 5-year CDS spikes to ~1,100bps as market fears AI bubble starts as credit event — kimmonismus · 2026-10-26
- AI demand reshapes cloud computing: scarce capacity and long-term deals replace on-demand — DavidLinthicum · 2026-10-26