OneLA Shares Linear-Attention States Across Beams for 1.54-2.46x Faster Generative-Rec Decoding
_reachsumit · x · 2026-09-14
- 生成式推荐依赖大束搜索生成数百候选,现有线性注意力推理系统需为每个 beam 物化完整循环状态或反复重放历史,内存与流量开销大。
- OneLA 用单一共享的 prompt 派生状态加上各 beam 的紧凑追加式分歧记录来表示全部 beam 状态,仅计算当步所需状态信息;配合轻量祖先索引追踪各 beam 历史,beam 更新无需复制已有记录,并用融合 GPU kernel 跨 beam 复用共享状态。
- 端到端解码加速 1.54-2.46 倍,同时大幅降低循环状态内存与数据搬运。
More from Infra
- Maia 200 hits ~12 TFLOP/s FP4 in 1mm²: density should be a first-class goal — thoefler · 2026-09-14
- Dev's 24/7 self-hosted AI stack: OpenWebUI, pidot, Tailscale, GLM and DeepSeek — andfanilo · 2026-09-14
- Dream Photonics' laser integration render caught mirroring the whole chip image — jwt0625 · 2026-09-14
- Engineer questions the inline PTX hype DeepSeek sparked: hand-written PTX isn't a proxy for performance — mike64_t · 2026-09-14
- Qwen3.8-Flash-Next on M3 Ultra: 559 t/s prompt processing, 31 t/s generation — rm-rf-rm · 2026-09-14
- New research: standard SGD matches AdamW for LLM RL training, with far less memory overhead — zhaoran_wang · 2026-09-14