DeepSeek V4.1 Flash joins AA-AgentPerf: GB300 hits 261M tokens/s per MW at 100 tok/s per user
ArtificialAnlys · x · 2026-10-14
Artificial Analysis added DeepSeek V4.1 Flash to its AA-AgentPerf agentic inference benchmark, with results across NVIDIA and AMD hardware.
Model highlights
- Scores 39 on the Intelligence Index, beating V4 Pro 0813's 36 with roughly a third of the parameters (552B backbone, 16B active vs 1.6T, 49B)
- New causal encoder-decoder architecture: 8B active per input token, 16B per output token, plus 196B Engram n-gram memory that can sit in host memory; compressed sparse attention cuts KV cache to about a quarter of the earlier V4 Flash
- Much cheaper to serve: runs on as few as 2 GPUs, with a smaller KV cache leaving room for more concurrent agents
Hardware rankings (100 tok/s per user)
- GB300 leads at 261M tokens/s per MW, ahead of B300 (220M), B200 (193M), GB200 (178M), and MI355X (128M)
- Single-agent output speed: Blackwell systems and MI355X reach 600–670 tok/s vs 372–392 tok/s on H100/H200
- V4.1 Flash delivers 1.3x (GB300) to 6.2x (B200) the throughput per MW of V4 Pro 0813
The benchmark replays real agentic coding sessions, including prompt caching and tool-call delays; the leaderboard now features DeepSeek V4.1 Flash, DeepSeek V4 Pro 0813, Kimi K3, and GLM-5.2/5.3.
More from Infra
- Agent traffic is a strange inference workload: long prefix caching beats raw tokens per second — rohanpaul_ai · 2026-10-14
- LightVAE speeds up video VAE decoding up to 2.24x on H100 with minimal quality loss — Shiqiao Gu · 2026-10-14
- Orbital data centers have no off-switch: Reddit debates AI safety paradox — Zappy_Oh · 2026-10-14
- Rednote's Dots unveils MSAR: 1.58x iso-loss compute speedup over residual baseline — teortaxesTex · 2026-10-14
- VirusTotal ships macOS Mirage sandbox to analyze fake password-prompt infostealers on Apple Silicon — jedisct1 · 2026-10-14
- ClickHouse merges IPCrypt support for GDPR-friendly IP pseudonymization — jedisct1 · 2026-10-14