Tencent Youtu's LightningMoBA speeds long-context attention 1.2-1.6x without quality loss

tencent · hf · 2026-11-20

Tencent Youtu Lab introduces LightningMoBA, co-designing selection and execution for Mixture of Block Attention via dynamic routing (thresholded per-query block scores emitting variable-cardinality sparse metadata), KV-centric ownership (grouping by query head/KV block, splitting high-fanout columns, reusing loaded KV), and runtime pipelining that overlaps K/V transfers with Tensor Core compute on Hopper GPUs. At matched Top-16 retention across five checkpoints, end-to-end generation accelerates 1.20-1.60x while mean RULER rises from 74.42 to 75.28.

Original post →

More from Infra

Infra channel →