Scaling DGX Spark clusters with QSFP cables for data-parallel training works nicely

austinvhuang · x · 2026-11-19

austinvhuang shares hands-on experience running data-parallel training on a cluster of NVIDIA DGX Spark units: memory bandwidth is poor for n=1 interactivity, but linearly scaling caveman-stupid data-parallel runs over QSFP cables is nice. He bought the units last year anticipating 'memorygeddon', and his bedtime ritual is now kicking off exploratory continued-pretraining runs before sleep — though the price point is harder to justify for a cluster today.

Related event: Linking Multiple DGX Sparks via QSFP for Data-Parallel Training(2 posts)→

Original post →

More from Infra

Infra channel →