YOCO Architecture Explained: How DeepSeek-V4.1-Flash and Gemma4 Squeeze KV Cache from the Layer Dimension
青稞AI · wechat · 2026-09-21
A detailed Chinese-language breakdown of YOCO (You Only Cache Once) and its variants. YOCO splits a decoder into a Self-Decoder that compresses context into one shared global KV cache and a Cross-Decoder whose layers all cross-attend to it, exploiting Prefill-Decode asymmetry and compressing KV cache along the layer dimension (vs GQA on entry and linear attention on sequence). Variants covered: YOCO-U (looped self-decoder + NOPE), YOCO-Sparse (shared sparse routing index). The piece explains DeepSeek-V4.1-Flash as YOCO/CED + CSA2 + SWA: 40 layers split 20+20, prefill only runs the encoder half, Global attention reuses KV across Full/Reindex/Reuse layers reading only Top-512 sparse KV, SWA KV kept in short-TTL DRAM with only the last 128 tokens replayed on loss, FP4 for global main KV. Also covers Gemma4 E2B/E4B's layer-wise KV reuse and WeLM's U-shaped KV-Mirror for prefill savings.
More from Models
- Why Is Inkling 2 Small Trained on So Few Tokens Yet Still Competitive? — georgejrjrjr · 2026-11-12
- OpenAI's five biggest failures laid out: pretraining decline, alignment fears, enterprise lost to Claude — teortaxesTex · 2026-11-12
- Fable 5.2 Hits 41.3% on Remote Labor Index, Up From 2.5% a Year Ago — alliekmiller · 2026-11-12
- Beaker automates MIT's weight-space programming: one-neuron edit lifts MATH-500 from 7% to 37% for under $4 — VictorSanh · 2026-11-12
- Researcher says Anthropic dodged questions on Mythos false positive rate; 37,966 findings, 2 exploited — OwariDa · 2026-11-12
- Grok spotted building meeting "Listen to Conversation" feature with live transcript — nima_owji · 2026-11-12