Baidu open-sources FlashMask: up to 1.55x faster than FlashAttention-4 for long-context training
BaiduResearch · x · 2026-10-16
Baidu Research released FlashMask, an open-source high-performance attention library for long-context LLM training. The paper FlashMask-3 was accepted at NeurIPS 2026.
Key features:
- Flexible support for complex attention masks via a column-wise sparse representation
- Up to 1.55× faster than FlashAttention-4
- Efficient context parallelism with load balancing and compute-communication overlap
- PaddlePaddle & PyTorch support via CuTe DSL, JIT-compiled kernels
- SM90 (Hopper) & SM100 (Blackwell) support
- Large head dimensions: 192/128, 256/256, 512/512, 576/512
More from Infra
- NVIDIA ships FP4-quantized Qwen3.8-Flash for local use, trending on Hugging Face — nvidia · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16
- DuoDiscover agents auto-build fast eval proxies, scoring 2.68x TPU kernel speedup — berkeley_ai · 2026-10-16
- Google Raises 2026 Capital Spending to Up to $205 Billion — emmanuelvivier · 2026-10-16
- Google open-sources AX, a Kubernetes-style orchestrator for billions of AI agent tasks — thisdudelikesAI · 2026-10-16
- Valtz v0.1.0.11 adds FLUX.3 support, local image gen up to 4K on Mac — TgoAI · 2026-10-16