DeepSeek-V4.1-Flash hits 200 tok/s on Vera Rubin as agent ships Engram-table sharing fix

woosuk_k · x · 2026-11-17

An engineer details optimizing DeepSeek-V4.1-Flash on NVIDIA Vera Rubin: $117.89 per chip-hour at DeepSeek off-peak pricing, 200 tok/s per user. As throughput grew, duplicated Engram tables ate KV cache memory and tanked prefix cache hit rates. An AI agent was tasked with fixing it and devised sharing Engram tables across the rack's GPU HBM via NVLink symmetric memory, with even lower lookup overhead — discovering, implementing and shipping the optimization on brand-new hardware with only a few human hints.

Original post →

More from coding & agent

coding & agent channel →