Exact Relative-Position Attention Kernels Deliver Up to 21x Backward Speedup at 8K Context
INSAIT-Institute · hf · 2026-11-09
INSAIT researchers propose an exact tiled attention algorithm that folds additive relative-position terms into FlashAttention-style computation via offset-aware tiling, product reuse across forward/backward passes, and offset-based gradient sharding. A unified Triton implementation yields 1.8–6.4x forward and 2.0–21x backward speedups at 8K-token sequences, cuts peak memory up to 128x over the strongest baseline per mechanism, and enables end-to-end long-context transcription 1.8x faster with 2.4x less memory — a drop-in replacement for relative-position transformers.
More from Infra
- Day 309/365 of GPU Programming: Optimizing LLaMA2-7B on Edge with SIMD and Multithreading — NandoDF · 2026-11-09
- Waymo explains at Hot Chips why camera-only autonomy plateaus at 99.9% — 4310sy · 2026-11-09
- Running a Home Mac mini as an Always-On Node for Remote AI Tasks — xiaohu · 2026-11-09
- GitHub investigates delayed Actions runs and webhook deliveries in new incident — DanielLockyer · 2026-11-09
- Hardware-aware LDM implementation cuts memory traffic via state unwinding and windowed writes — SonglinYang4 · 2026-11-09
- AMD surges 11% to $449, extends gains in premarket trading — firstadopter · 2026-11-09