DeepSeek-V4.1-Flash hits 200 tok/s on Vera Rubin as agent ships Engram-table sharing fix
woosuk_k · x · 2026-11-17
An engineer details optimizing DeepSeek-V4.1-Flash on NVIDIA Vera Rubin: $117.89 per chip-hour at DeepSeek off-peak pricing, 200 tok/s per user. As throughput grew, duplicated Engram tables ate KV cache memory and tanked prefix cache hit rates. An AI agent was tasked with fixing it and devised sharing Engram tables across the rack's GPU HBM via NVLink symmetric memory, with even lower lookup overhead — discovering, implementing and shipping the optimization on brand-new hardware with only a few human hints.
More from coding & agent
- Indie dev: abundant tokens let AI agents run the whole video pipeline end-to-end — yihui_indie · 2026-11-17
- Anthropic Engineers Share 24-Minute Walkthrough of Little-Known Claude Code Features — goyalshaliniuk · 2026-11-17
- Open-source agent-browser launches: browser & desktop automation built for AI agents that verify their own work — evilrabbit_ · 2026-11-17
- Ripple-65B hackathon: model builds its own IDE with <80ms autocomplete, 300 tok/s — Azaliamirh · 2026-11-17
- LM Studio launches Agent Remote Control for running agents on any device — MatthewBerman · 2026-11-17
- Qwen 3.8 27B at 300-500 tok/s on a 5090 laptop, tuned for parallel agents — tebjan · 2026-11-17