Open-Source LSA Sparse Attention Extends Qwen 27B to 10M-Token Context

Prefrontal AI, working with NVIDIA and Nebius, open-sourced Latent Sparse Attention (LSA), which sparsifies Qwen 3 27B's attention with negligible accuracy loss, extends context to 10 million tokens, and speeds up prefill by up to 72.4x. The release includes SGLang integration, context parallelism, and H200/B300 kernels.

2026-11-03 ~ 2026-11-03 · 3 related posts

• Prefrontal open-sources LSA: 72.4x faster attention prefill at 10M tokens, accuracy on par — jcrwhittington · 2026-11-03
• Fully open release: SGLang integration, H200/B300 kernels and all training details published — francoisfleuret · 2026-11-03
• Open-source sparse attention: Qwen 27B with no accuracy loss, 10M-token context — Immediate_Sky_6566 · 2026-11-03