DeepSeek V4.1-Flash hits 5,800 TPS per Ascend 950DT card, but critics call throughput underwhelming
teortaxesTex · x · 2026-10-04
- A developer reports deploying DeepSeek-V4.1-Flash with mixed FP8-FP4 precision on 32 Huawei Ascend 950DT cards: single-card decoding throughput reaches 5,800 TPS at 13.79 ms for 128K-sequence scenarios, while a low-latency config keeps latency under 5 ms at 2,727 TPS.
- teortaxesTex argues the results are weak: the throughput is only about one-third of what DeepSeek reportedly achieved per GPU with V4-Flash in the DSpark report, despite TileLang kernels. He suggests a larger deployment unit or future optimizations (e.g. npugraphex) may be needed.
More from Infra
- Nvidia down ~40% from October peak as Anthropic IPO pops 34% above expectations — The Verge AI · 2026-11-20
- Ion Stoica explains why complex AI workloads increasingly fit Ray's design — PyTorch · 2026-11-20
- GPU rents up, token prices down: inference providers' break-even load jumps 22%–75% in 5 months — AccBalanced · 2026-11-20
- Missing config file was throttling GPUDirect Storage — fix yields 3x reads, 9x writes — AccBalanced · 2026-11-20
- NVIDIA Dynamo cuts RL rollout bottlenecks with KV-aware routing and delta weight updates — NVIDIAAI · 2026-11-20
- Arm reaches the AI laptop: Surface Laptop Ultra runs 120B models locally — kimmonismus · 2026-11-20