DeepSeek Flash Hits ~1000 TPS, With Models Doing the Kernel Optimization
saranormous · x · 2026-11-05
DeepSeek v4.1 flash's fast endpoint reaches 991 output tokens/s on Artificial Analysis, on the Pareto frontier for lowest time per task. The team used an internal harness to optimize 80+ kernels and bottlenecks, covering kernel fusion, programmatic dependent launch, and more.
The bigger story is model-driven model infrastructure: profiling, rewriting, and benchmarking are well-defined and verifiable — ideal hill-climbing tasks for agents. An early checkpoint of Kimi k3 reportedly handled the majority of kernel optimization for its own later development, and models are now orders of magnitude better than humans at kernel work.
More from Infra
- Splitting one 27B across two 3090s was slow: one model per GPU hits 243 tok/s combined — StewartalsopIII · 2026-11-06
- Shawn Hymel's new tutorial: using Meshtastic devices as IoT sensor nodes with Python — ShawnHymel · 2026-11-06
- UBS: Aspeed Has Secured 2027 BMC Orders on Strong Agentic AI Server Demand — BenBajarin · 2026-11-06
- Firecracker VM escape disclosed: researchers claim root execution on the host — moyix · 2026-11-06
- Mercedes partners with Liquid AI to run models on-device in cars, not the cloud — JosephJacks_ · 2026-11-06
- Liquid AI's LFM3 hailed as the iPhone moment for local AI on consumer hardware — JosephJacks_ · 2026-11-06