Prefrontal open-sources LSA: 72.4x faster attention prefill at 10M tokens, accuracy on par
jcrwhittington · x · 2026-11-03
Prefrontal AI (with NVIDIA and Nebius) released Latent Sparse Attention (LSA), a step toward subquadratic attention that matches full-attention accuracy at ultra-long context.
- How it works: keys are hierarchically clustered in a learned latent space by meaning rather than position; each query scores the hierarchy of clusters first, then drills into the most relevant ones—avoiding the key search that takes 98% of attention-layer time at 10M tokens
- Two variants: D-LSA attaches to pre-trained models via a short distillation (only the new indexer is trained); N-LSA is trained natively from scratch
- Results (10M tokens, H200): D-LSA speeds attention-layer prefill 72.4x over full attention on Qwen-3.8-27B (9.2x end-to-end) and 22.7x over GLM-5.3-Flash's own sparse attention (2.2x end-to-end), with decode speedups of 4.6x/42.5x at parity accuracy; training a hybrid model from scratch takes 1.24x less wall-clock time
- Fully open: code with SGLang integration and context parallelism, H200/B300 kernels, all training/inference details (paper arXiv:2610.42147)
More from Infra
- Google's Suncatcher satellite runs Gemma 270mi n space—power limits mean only one TPU lit at a time — demian_ai · 2026-11-03
- Mistral VP: Le Chonk trained on just 10 MW (~4,000 H100s), begs investors for 1 gigawatt — _AndrewZhao · 2026-11-03
- UCSB awards IEE seed funding to three ECE projects on energy-efficient AI and quantum hardware — ninamiolane · 2026-11-03
- Europe's Physical AI Agenda: Gigafactories, Sovereign Compute and AI Act Rules — ingliguori · 2026-11-03
- Weight teleportation open-sourced: cross-datacenter RL weight updates under 0.23s — gharik · 2026-11-03
- YouTuber's "F-You to Sam Altman" video showcases a local AI GPU rig — unchikuso · 2026-11-03