DeepSeek V4.1 Flash's global attention: compressed attention with three KV-reuse variants
nrehiew_ · x · 2026-09-11
nrehiew explains DeepSeek V4.1 Flash's "global attention": an updated compressed-attention scheme. All three variants run an indexer over smaller keys/values, do selection on the main KV, concatenate with windowed KV, then attend. Variants differ in:
- no reuse
- reusing KV from earlier layers and indexer K
- reusing the index selection output
The 20-layer encoder + 20-layer decoder design cuts KV cache in half: the bottom half builds a KV cache shared with upper half layers via per-layer projection — inspired by YOCO, hence "decoder-decoder" (SWA layers excluded).
Related event: DeepSeek V4.1 Flash Deep Dive: KV Cache Compression at the Frontier(9 posts)→
More from Models
- Anthropic Accuses Moonshot of Routing 300K User Queries to Claude via 5,380 Fake Accounts — toptickcrypto · 2026-09-11
- DeepSeek 4.1 flash reportedly uses large ngram embeddings, echoing Qwen4 architecture — ccerrato147 · 2026-09-11
- ValsAI launches RSI Index, first third-party benchmark measuring how close AI is to self-improvement — JenniferHli · 2026-09-11
- Assistant Benchmark goes live: 61 assistants scored across 15 real-use dimensions — Scobleizer · 2026-09-11
- Devin's New Model Verdict: Not a Benchmaxxer, a 'Killer Execution Model' at $20/Month — brandon_galang · 2026-09-11
- Business Insider Asked ChatGPT, Gemini, Claude and Grok How AI Could End Humanity — coinfanking · 2026-09-11