Leaked DeepSeek V4.1-Flash report: 552B MoE, 1M context, KV cache cut to ~1/40
teortaxesTex · x · 2026-11-03
Screenshots circulating show an unverified 51-page tech report titled "DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression".
- A 552B-parameter multimodal MoE with a Causal Encoder-Decoder (CED) architecture, supporting contexts up to 1M tokens
- Only 1.68B parameters activated per token during decode; cross-layer KV cache reuse (CSA) with FP4 shrinks global KV cache to 680 bytes/token — roughly 1/40 of V4.1
- Persistent KV cache before SWA boundary is offloaded to SSD/host memory, 1/8 that of V4.1-Flash
- Report claims pretraining on 4.51T multimodal tokens with substantially better-than-baseline performance
A second screenshot shows an agent (286 steps, $0.59) finding and fixing a real design flaw in a paper-reader plugin: papers lived only in IndexedDB, so the Ask action sent a title + quote with no evidence the agent could open. The fix: persist every uploaded paper as a first-class Host artifact with extracted text and give the agent a read tool. The poster's verdict: "It works, but it is a bit naive."
More from Models
- Developer warns: do NOT use weight-space programming — djcows · 2026-11-03
- Musk: diffusion wins on any bitstream; CoBit applies one diffusion recipe to text, speech, images, proteins — LucaAmb · 2026-11-03
- New Kaggle benchmark transcribes 1621–1737 Portuguese records; Gemini 4 Argon leads at 0.86 — mariofilhoml · 2026-11-03
- Yoav Goldberg speculates RL training included tasks on interpreting and editing model weights — yoavgo · 2026-11-03
- Baseten launches SubQ Snap 1.1: a 384K-context 'System 1' decision model tied for #1 on Decision Index — baseten · 2026-11-03
- Open-weights licenses tighten: Qwen, Kimi, GLM and Step add provider conditions, DeepSeek stays pure MIT — teortaxesTex · 2026-11-03