DeepSeek V4.1 Flash gets up to 1.5x faster on GB300 NVL72 in one month
ArtificialAnlys · x · 2026-11-07
Artificial Analysis's AA-AgentPerf benchmark, which replays real coding-agent trajectories on datacenter systems, shows DeepSeek V4.1 Flash speeds on GB300 NVL72 improved up to 1.5x per agent in a month:
- Single agent: 848 tok/s, up from 596 (1.42x)
- Throughput: 1.6x per GPU at 80 tok/s per agent
- Gains come from a newer vLLM version, new vLLM kernels for DeepSeek V4.1 on GB300, FlashInfer autotuning, fuller CUDA graph coverage, tuned speculative decoding, and co-located prefill/decode in NVIDIA's updated Dynamo + vLLM stack
A clear demonstration that inference software-stack optimization can yield hardware-scale gains within weeks.
More from Infra
- Huawei Ascend 960 roadmap pulled forward up to 3 quarters; analyst says scale-up domain beats efficiency — teortaxesTex · 2026-11-07
- CroissAttn runs all six attention matmuls in FP4 on Blackwell: 1.5× throughput, 55% less memory — TristanThrush · 2026-11-07
- SemiAnalysis: SGLang on AMD Mi355 crushes B200/B300 in throughput test — AnushElangovan · 2026-11-07
- Musk explains Terafab: not 'terror fab' but a terawatt-scale fab — elonmusk · 2026-11-07
- AMD launches free AI Developer Program with cloud credits and ROCm training — AnushElangovan · 2026-11-07
- Musk says data center unpopularity is temporary: Memphis is booming with 2x wages — inductionheads · 2026-11-07