BAAI's CoWA Attention Speeds Up Long-Context Inference

BAAI proposes CoWindow Attention (CoWA), which distributes remote context across KV heads via complementary windows to eliminate redundant full-causal computation. It achieves 7.4x faster forward pass on 128K context and saves 28.5% training FLOPs.

2026-09-29 ~ 2026-09-29 · 2 related posts

• BAAI's CoWA attention cuts training latency 7.4x while matching FullAttn quality to 32B — BAAI · 2026-09-29
• CoWA and MALA attention: 7.4x faster forward pass at 128K, 28.5% fewer training FLOPs — BitExternal4608 · 2026-09-29