Inkling too small to saturate Tensor Cores; BBBK speedups scale with model size

andrew_n_carr · x · 2026-11-16

In a quote-tweet, Andrew Carr highlights liyanc98's explanation of BBBK optimization on Inkling: the small model is too small to saturate Tensor Cores and little time was spent optimizing smaller shapes, so speedups scale with model size — a hypothetical three-month training run would see far larger overall time savings.

Original post →

More from Infra

Infra channel →