Zyphra Unveils ZHA, a Tree-Structured Subquadratic Attention

Zyphra introduced ZHA, a hierarchical tree-structured sparse attention method that achieves subquadratic complexity in both prefill and decoding, with claimed speedups of up to 400x on long contexts.

2026-11-03 ~ 2026-11-03 · 3 related posts

• ZHA achieves subquadratic prefill and decode, converts existing models via distillation — BerenMillidge · 2026-11-03
• Zyphra Introduces Hierarchical Attention, Claiming up to 400x Speedups at 1B-Token Contexts — teortaxesTex · 2026-11-03
• Zyphra's ZHA brings subquadratic attention via hierarchical tree-structured indexer — yacineMTB · 2026-11-03