YOCO Architecture Explained: How DeepSeek-V4.1-Flash and Gemma4 Squeeze KV Cache from the Layer Dimension
青稞AI · wechat · 2026-09-21
A detailed Chinese-language breakdown of YOCO (You Only Cache Once) and its variants. YOCO splits a decoder into a Self-Decoder that compresses context into one shared global KV cache and a Cross-Decoder whose layers all cross-attend to it, exploiting Prefill-Decode asymmetry and compressing KV cache along the layer dimension (vs GQA on entry and linear attention on sequence). Variants covered: YOCO-U (looped self-decoder + NOPE), YOCO-Sparse (shared sparse routing index). The piece explains DeepSeek-V4.1-Flash as YOCO/CED + CSA2 + SWA: 40 layers split 20+20, prefill only runs the encoder half, Global attention reuses KV across Full/Reindex/Reuse layers reading only Top-512 sparse KV, SWA KV kept in short-TTL DRAM with only the last 128 tokens replayed on loss, FP4 for global main KV. Also covers Gemma4 E2B/E4B's layer-wise KV reuse and WeLM's U-shaped KV-Mirror for prefill savings.
More from Models
- Blogger bets OpenAI will make AI feel real-time within four months — imjustnewatai · 2026-09-21
- Chinese open models reportedly just 4.4 months behind US frontier models — emmanuelvivier · 2026-09-21
- StepFun unveils Step 5 Preview: 600B-param MoE with 1M-token context — emmanuelvivier · 2026-09-21
- MiMo near-SOTA on DeepSWE with just ~$2.6M RL run: will data cost more than training? — my_cat_can_code · 2026-09-21
- humansand's Persimmon model learns to share info gradually like humans, with Trickle Test — niloofar_mire · 2026-09-21
- Why yes/no answers are fast for LLMs: output tokens dominate latency — tinyfool · 2026-09-21