antirez demos DGX Spark streaming a half-trillion-parameter model from SSD, "kinda usable for QA"
antirez · x · 2026-09-14
antirez shared a demo of a DGX Spark executing a half-trillion-parameter model via SSD streaming — "kinda usable for QA," though not for coding unless you let it run overnight. He admits README prefill is still much slower than it should be compared to Metal, an implementation issue he plans to fix in a next commit. A rare first-hand look at memory-streamed on-device inference of very large models from a high-profile developer.
Related event: antirez Runs 500B-Param DeepSeek on a Single DGX Spark via SSD Streaming(3 posts)→
More from Infra
- Clover-35B-A3B Quantized to 29.4 GB Runs at ~42 tok/s on a 64 GB Mac — viveky259 · 2026-10-31
- Liquid Agent brings fully local LFM3 agent to Linux with one-command install — JosephJacks_ · 2026-10-31
- A fully local office AI stack: MiniCPM, Qwen3, LFM2.5 and Whisper — HowDevelop · 2026-10-31
- a16z: GPU rental prices up 106% in 3 months while token prices fall 58% — a16z · 2026-10-31
- vLLM nightly adds PLE disk offload, with a telling [Qwen4Exp] tag on the PR — rmhubbert · 2026-10-31
- Solo AI studio: per-minute GPU rentals rendered a 328-frame Blender shot for ~$1 — r0ck3t23 · 2026-10-31