Inkling Released with 15% Speed Boost
LysandreJik · x · 2026-07-16
Thinking Machines has launched Inkling: an open large model supporting text, image, and audio inputs with text output, scaled at 1T.
The post highlights the pluggable optimization space for inference acceleration:
- Replacing the model's causal Conv1D with the causal-conv1d kernel requires just a one-line change, boosting throughput by roughly 4% tokens/s.
- Swapping the attention implementation to FlashAttention-4 adds another 11% improvement.
- Combined, overall throughput increased by 15% without altering the architecture or retraining.
- The author notes there is still significant room for optimization in the MoE layers.
The core takeaway is that the kernels approach allows model developers to quickly integrate better implementations just like changing a config, while kernel developers focus on delivering the fastest versions, ultimately benefiting everyone.
More from Infra
- Should LLM tokens carry green data-center validation labels, like Fair Trade? — jdavid · 2026-09-03
- Commentary: American construction workers want data centers, not just the grey curve — saranormous · 2026-09-03
- Six load forecasters benchmarked on GPU-hours: none beat the last-value baseline — Vegetable-Top-3670 · 2026-09-03
- Visited a 240MW AI data center in Richmond, VA — surprisingly quiet, no high-pitch noise — AndyMasley · 2026-09-03
- Carmack revives rotovators: spinning tethers could slash the cost of space-based data centers — ID_AA_Carmack · 2026-09-03
- GLM-5.3-Flash beats DeepSeek-V4-Flash for writing and vision on 2× DGX Spark — kuhunaxeyive · 2026-09-03