Meta's TLX Jagged Flash Attention: 3.2K lines of Triton, beats FA4 by 13% forward / 50% backward
lmoroney · x · 2026-10-02
Meta published Jagged Flash Attention kernels for Blackwell built on TLX (Triton Low-level Extensions), with code public on GitHub.
- The TLX kernel is 3.2K lines of Triton-level code, roughly 3x shorter than the 10K-line CuteDSL FlashAttention-4 kernels.
- On the jagged shapes used in Meta's ads models, it beats FA4 by 13% on forward and 50% on backward pass.
- The takeaway for ML systems learners: high-level Triton with the right hardware hooks can deliver competitive kernels that modeling engineers can actually read and extend—performance and teachability can coexist.
Related event: Meta's 3.2K-line Triton JFA Kernel Beats FA4 on Blackwell(2 posts)→
More from Infra
- Gridline launches one-API fine-tuning, distillation and batch inference for open models — Scobleizer · 2026-11-19
- Ascend output could hit ~20M chips next year; memory and packaging, not logic, are the bottleneck — teortaxesTex · 2026-11-19
- AMD Loom boosts Qwen3.5-4B prefill up to 2.6x on Radeon 890M laptops — jfowers_amd · 2026-11-19
- Dev submits PR adding periodic checkpoints to GKE sandboxes so agents don't lose progress — yoimnotkesku · 2026-11-19
- AWS Lambda Adds GPU Support via Managed Instances for Inference and Media Workloads — DavidWells · 2026-11-19
- Nvidia Allegedly Made GPUs Artificially Scarce to Keep Prices Sky-High — GaryMarcus · 2026-11-19