DeepSeek-V4.1-Flash Hits 63x Throughput on Vera Rubin, With AI Agent Doing the Optimization

woosuk_k · x · 2026-11-17

The vLLM team published first DeepSeek-V4.1-Flash results on NVIDIA Vera Rubin NVL72, running Dynamo + vLLM on SemiAnalysis's AgentX benchmark. Versus GB300 NVL72 on the same stack, per-chip total token throughput is 63.4x higher at 400 tok/s/user and 93.3x at 600 tok/s/user.

Optimizer @liujuntian notes that as throughput rose, prefix cache hits dropped because duplicated Engram tables were eating KV cache memory. He tasked an AI agent with the fix; it shared Engram tables across the rack's GPU HBM via NVLink symmetric memory, with even lower lookup overhead. He says the agent could discover, implement and ship optimizations on brand-new hardware with only a few human hints.

Related event: vLLM Reveals DeepSeek V4.1 Flash Results on Vera Rubin NVL72(3 posts)→

Original post →

More from coding & agent

coding & agent channel →