Sparse attention as multilevel retrieval: the DeepSeek V3.2 trick mirrors search ranking stacks
nptacek · x · 2026-09-11
cosminnegruseri's thread on sparse attention: DeepSeek V3.2's approach — low-precision full attention to select top-k, then full-precision attention on those — mirrors two-stage search systems (recall + ranking). Sparse attention can itself be multilevel, like static rank, multi-index retrieval, lightweight and full reranking, with different layers handling precision vs recall. On hybrids: RNNs do state tracking, attention does retrieval, yielding much smaller KV caches and better compute, though training may be harder. He also cites a Jeff Dean interview endorsing hierarchical approaches for next-gen LLM systems.
More from Models
- ToneBench eval: GPT-6.1 Terra ranks #23 at writing — cheaper, not better, than GPT-5.6 Terra — OnlyProggingForFun · 2026-10-28
- Cursor user complains it silently switches to Grok, which fails basic tasks — kylegawley · 2026-10-28
- Musk says Grok Bot keeps improving, calls Vault feature a game changer — beffjezos · 2026-10-28
- Claude refuses to gather legitimately public information, sparking guardrail backlash — mkheck · 2026-10-28
- DesignArena launches Code to Video Arena: models direct 15-45s films purely in code — yshan2u · 2026-10-28
- ChatGPT's 'yes, but with an important clarification' often precedes pure nonsense — Crixusgannicus · 2026-10-28