Zyphra unveils Hierarchical Attention claiming 35x speedup at 10M tokens
teortaxesTex · x · 2026-11-05
Zyphra introduces Hierarchical Attention (ZHA), a path to true subquadratic attention that searches a learned tree of compressed summaries—projected 35x speedup over dense attention and 5x over MSA at 10M tokens, growing to 400x/55x at 1B. Quoting author Jonathan Chang is skeptical of fixed top-p at long context, arguing noise grows with context and dynamic shapes hurt efficiency; a fixed top-k with a schedule may suffice.
More from Research
- Claude drifts from 🌀 to 💖: refined AI symbols become less legible to humans (67%→44%) — mimi10v3 · 2026-11-05
- Engram: brief training fixes long-run models' increasingly inscrutable notes — scott_linderman · 2026-11-05
- JITEngine synthesizes custom LLM inference engines, 2.7x faster on Apple Silicon — CShorten30 · 2026-11-05
- NMSLIB author publicly slams new vector search preprint for downplaying prior work — srchvrs · 2026-11-05
- NMSLIB author: new preprint cites us to back an unsubstantiated claim — srchvrs · 2026-11-05
- Foresight-SWE benchmark: coding agents solve 60.7% of issues but only 14.8% without introducing bugs — OfirPress · 2026-11-05