Sparse attention as multilevel retrieval: the DeepSeek V3.2 trick mirrors search ranking stacks
nptacek · x · 2026-09-11
cosminnegruseri's thread on sparse attention: DeepSeek V3.2's approach — low-precision full attention to select top-k, then full-precision attention on those — mirrors two-stage search systems (recall + ranking). Sparse attention can itself be multilevel, like static rank, multi-index retrieval, lightweight and full reranking, with different layers handling precision vs recall. On hybrids: RNNs do state tracking, attention does retrieval, yielding much smaller KV caches and better compute, though training may be harder. He also cites a Jeff Dean interview endorsing hierarchical approaches for next-gen LLM systems.
More from Models
- DeepSeek 4.1 flash reportedly uses large ngram embeddings, echoing Qwen4 architecture — ccerrato147 · 2026-09-11
- ValsAI launches RSI Index, first third-party benchmark measuring how close AI is to self-improvement — JenniferHli · 2026-09-11
- Assistant Benchmark goes live: 61 assistants scored across 15 real-use dimensions — Scobleizer · 2026-09-11
- Devin's New Model Verdict: Not a Benchmaxxer, a 'Killer Execution Model' at $20/Month — brandon_galang · 2026-09-11
- Business Insider Asked ChatGPT, Gemini, Claude and Grok How AI Could End Humanity — coinfanking · 2026-09-11
- Claims resurface that Moonshot's Kimi distilled from Claude raw CoTs — xuanalogue · 2026-09-11