DeepSeek-V4.1-Flash redesigns the Transformer for agents, cutting KV cache to 890 bytes/token
AndLukyane · x · 2026-09-21
- What it is: DeepSeek-V4.1-Flash rethinks the Transformer around agent workloads — coding, research, and tool-using agents process hundreds of thousands of tokens while emitting little output, making prefill compute and KV cache the key infrastructure bottleneck.
- Architecture: A 552B-parameter MoE backbone with a new Causal Encoder-Decoder (CED) design, splitting 40 layers into a 20-layer causal encoder and 20-layer decoder; only 8B parameters activate per token during prefill and 16B during decoding.
- Cache compression: Compressed Sparse Attention 2 (CSA2) shares KV representations and sparse-attention decisions across layers; SWA Bounded Replay avoids persistently storing sliding-window KV states; FP4 KV quantization shrinks the global cache to just 890 bytes per token — about a quarter of DeepSeek-V4-Flash's.
- Capabilities retained: 1M-token context, native vision, and strong agentic behavior. Trained on 45T multimodal tokens, with much of the gain coming from scaling auto-generated agent tasks, environments, and rollouts.
More from Infra
- Komlós conjecture solution announced, with overlooked implications for neural network quantization — stevenstrogatz · 2026-09-21
- Nvidia Names 5 Companies Using AI for Clean Energy, Grid Reviews Cut From 45 Days to 2 Minutes — nordicinst · 2026-09-21
- Mozilla AI runs a local 30B model end-to-end to open a real bugfix PR, fully offline — mozilla-ai · 2026-09-21
- NVIDIA spotlights 5 AI clean-energy companies, grid review cut from 45 days to 2 minutes — NVIDIA Blog · 2026-09-21
- Cohere Labs launches Local AI community program for local inference and hardware tuning — Cohere_Labs · 2026-09-21
- Gewell: custom Gemma 4 inference engine cuts KV cache VRAM to 0.625x, losslessly — stoppableDissolution · 2026-09-21