DeepSeek docs reveal cross-datacenter inference tradeoffs: fiber 'easily severed by construction'
teortaxesTex · x · 2026-10-21
DeepSeek's open-sourced V4.1 Flash inference deployment doc discloses a striking detail: the model's heavy KV cache compression makes splitting Prefill and Decode across data centers technically feasible, at a reliability cost—the fiber between campuses is "easily severed by construction work." The doc recommends 400 Gbps of interconnect per 8 GPUs; in practice DeepSeek's inference nodes sit in different but adjacent data centers.
Other engineering details:
- Storage servers: 20× 30.73TB SSDs each, ≥40GB/s read and ≥5GB/s write, sized for 24 hours of data
- 14.4TB logical storage per GPU, physical capacity larger
- 3FS uses 10+2 erasure coding, tolerating two simultaneous node failures per EC group; reads hit all 11 nodes, any 10 suffice
- A GC Worker reclaims expired KV cache via LRU to keep disk usage below a watermark
Related event: DeepSeek Details V4.1 Flash Inference Deployment: Engineering, Not Magic(2 posts)→
More from Infra
- Hugging Face TRL officially supports AMD GPUs, full test suite runs on MI300X per PR — QGallouedec · 2026-10-21
- GPUs for prefill, custom silicon for decode: General Compute bets on Positron heterogeneous inference — rohanpaul_ai · 2026-10-21
- NVIDIA GTC 2026 kicks off in Berlin with supply and volatility on stage — kimmonismus · 2026-10-21
- Data center power set to double by 2030 as grid queues top 2,500 GW — ingliguori · 2026-10-21
- Interactive chart plots Artificial Analysis Intelligence Index against model parameter count — xrailgun · 2026-10-21
- antirez tries DHH's Omarchy on a Dell XPS, plans to adapt DwarfStar SSD model streaming — antirez · 2026-10-21