ThunderKittens Lands on Vera Rubin: NVFP4 GEMM Tops 22 PFLOPS
Together AI's kernel team obtained early access to NVIDIA's Vera Rubin NVL72 system, studied the new ISA within days, and ported ThunderKittens NVFP4 and FP8 GEMM kernels to Rubin GPUs, achieving over 22 and 12 PFLOPS respectively—comparable to official solutions like cuBLAS and CUTLASS. The team published a technical blog series detailing the porting process, among the first public third-party Rubin kernel optimization efforts.
Confirmed
- NVFP4 GEMM reached 22 PFLOPS (up to 22.4 PFLOPS at 16k scale) and FP8 reached 12 PFLOPS, on par with cuBLAS and CUTLASS.
- The blog starts from the original B200 GEMM and incrementally adds Rubin features: wider MMA steps, more tensor/shared memory, B-side collectors, and early A release.
- The team notes these kernels are still early-stage, with further optimizations such as LUT yet to come.
Why it matters
- The team highlights that Rubin's biggest change is tensor cores consuming operands twice as fast, but wider MMA alone cannot feed the cores—only combined improvements in data reuse and deeper pipelining pushed the 16k NVFP4 GEMM to 22.4 PFLOPS, offering direct guidance for other developers adapting to Rubin.
- The authors state that Rubin's performance ceiling remains far from fully tapped, with more optimizations to expect.
2026-09-11 ~ 2026-09-11 · 7 related posts
Primary sources
- ThunderKittens lands on NVIDIA Vera Rubin, pushing NVFP4 GEMMs past 22 PFLOPS — togethercompute ·
- Feeding Rubin's tensor cores took more than wider MMAs: reuse and deeper pipelines — togethercompute ·
- Together AI's Rubin kernel blog builds from B200 GEMMs, stacking new ISA features — togethercompute ·
- [source] ThunderKittens lands on NVIDIA Vera Rubin, pushing NVFP4 GEMMs past 22 PFLOPS — togethercompute · 2026-09-11
- [source] Feeding Rubin's tensor cores took more than wider MMAs: reuse and deeper pipelines — togethercompute · 2026-09-11
- [source] Together AI's Rubin kernel blog builds from B200 GEMMs, stacking new ISA features — togethercompute · 2026-09-11
- Together Compute ships early Rubin GPU kernels, unlocking wider MMA steps and new memory features — togethercompute · 2026-09-11
- ThunderKittens adds NVIDIA Vera Rubin GPU support ahead of next-gen inference — simran_s_arora · 2026-09-11
2 near-duplicate retellings: simran_s_arora · HazyResearch