Inkling too small to saturate Tensor Cores; BBBK speedups scale with model size
andrew_n_carr · x · 2026-11-16
In a quote-tweet, Andrew Carr highlights liyanc98's explanation of BBBK optimization on Inkling: the small model is too small to saturate Tensor Cores and little time was spent optimizing smaller shapes, so speedups scale with model size — a hypothetical three-month training run would see far larger overall time savings.
More from Infra
- WSJ ran 66 Nvidia/Jensen headlines in 4 days, joke 'benchmark' shows — xiaosun86 · 2026-11-16
- SemiAnalysis: renting US compute to China yields Nvidia ~$7M profit per megawatt — xiaosun86 · 2026-11-16
- Reuters: Nvidia chips pile up in Hong Kong as China withholds import approval — _weiping · 2026-11-16
- Tongxin Intelligent raises hundreds of millions RMB to build AGI-native inference chips — 量子位 · 2026-11-16
- CMU's Singularity-Bench: Opus 5.5 builds AI infra beating SGLang, scores 1.25 vs human baseline 1.0 — 新智元 · 2026-11-16
- GLM-5.3-Flash on two Mac Studios hits ~140 tokens/s via tensor parallelism — cryotic · 2026-11-16