DeepSeek job posting reveals 100k-accelerator cluster challenges amid Ascen 950 training rumor

teortaxesTex · x · 2026-11-20

A DeepSeek job posting for supercomputing cluster ops reveals stark challenges: making a 100,000-accelerator cluster as stable as a single computer (near 300 failures to handle daily), tuning new nodes to peak performance out of the box, building observability for GPU utilization/NCCL latency/checkpoint signals, and automating cluster self-healing so on-call engineers sleep through the night. The poster notes reliably running such clusters is something DeepSeek — and nobody in China — has done with these chips; getting hardware is only step one. A quoted tweet claims (unverified) that DeepSeek V5 will train entirely on Huawei Ascend 950, despite multi-fold costs.

Original post →

More from Infra

Infra channel →