Scaling DGX Spark clusters with QSFP cables for data-parallel training works nicely
austinvhuang · x · 2026-11-19
austinvhuang shares hands-on experience running data-parallel training on a cluster of NVIDIA DGX Spark units: memory bandwidth is poor for n=1 interactivity, but linearly scaling caveman-stupid data-parallel runs over QSFP cables is nice. He bought the units last year anticipating 'memorygeddon', and his bedtime ritual is now kicking off exploratory continued-pretraining runs before sleep — though the price point is harder to justify for a cluster today.
Related event: Linking Multiple DGX Sparks via QSFP for Data-Parallel Training(2 posts)→
More from Infra
- Datadog and Baseten fine-tune Qwen 3.5 9B for a 50x faster LLM summarizer — baseten · 2026-11-19
- Nvidia posts $116B quarter as analyst moves rating toward risk over unresolved uncertainties — BenBajarin · 2026-11-19
- Oracle co-CEO admits 'classic mistake' selling data centers to towns — jeremyakahn · 2026-11-19
- Surface Laptop Ultra With NVIDIA RTX Spark Runs 94GB Model, Finishes 4+ Enterprise Agent Workflows Per Hour — ryanshrout · 2026-11-19
- Nace ships 71.8GB DeepSeek V4.1 Flash coding quant with 39% lower KL than rivals — m4rkGS · 2026-11-19
- Dev open-sources self-hosted model registry so inference workers boot when HF is down — cmm324 · 2026-11-19