O(NlogN) tree-based attention retains 97% accuracy on long-context MQAR benchmark
Alarming-Emotion-894 · reddit · 2026-10-11
A Reddit post introduces ALHR (Adaptive Learnable Hierarchical Routing), a static binary-tree attention system using learnable functions to reduce the number of keys involved. It achieves O(NlogN) complexity, retains 97% accuracy on the long-context MQAR benchmark, and scales far better in VRAM as tokens grow.
Related event: ALHR: Binary-Tree Sparse Attention Achieves O(NlogN) Long-Context Inference(2 posts)→
More from Research
- Inferring goals from failure: online Bayesian goal inference for boundedly-rational agents — xuanalogue · 2026-10-11
- Alignment researcher points to Rohin Shah's value learning sequence and IRL model misspecification — xuanalogue · 2026-10-11
- Looped LM paper: 1.6B model matches full-cache baseline with 3x smaller KV cache — rupspace · 2026-10-11
- Pure RL discovers superhuman robot strategies in sim, transfers zero-shot to real hardware — KyleMorgenstein · 2026-10-11
- Softmax picks probabilities, cross-entropy picks the target: a 3-class walkthrough — techNmak · 2026-10-11
- PartLLM brings LLM-powered 3D mesh part segmentation to SIGGRAPH Asia with code released — Promptmethus · 2026-10-11