FlashInfer Merges Blackwell Kernels, Boosting E2E Inference Speed by 7.6%

YouJiacheng · x · 2026-08-06

FlashInfer has officially merged the tinygemm2 kernels optimized for the Blackwell architecture (SM100/SM103).

Performance: Delivers up to 1.79x faster speeds on large shapes, with an 18-23% geomean reduction in kernel time.
End-to-End Speedup: Translates to up to a 7.6% end-to-end serving performance boost in SGLang.
Compatibility: Ensures bit-identical outputs compared to the reference implementation and gracefully falls back on unsupported architectures.

More from Infra