Inkling Released with 15% Speed Boost
LysandreJik · x · 2026-07-16
Thinking Machines has launched Inkling: an open large model supporting text, image, and audio inputs with text output, scaled at 1T.
The post highlights the pluggable optimization space for inference acceleration:
- Replacing the model's causal Conv1D with the causal-conv1d kernel requires just a one-line change, boosting throughput by roughly 4% tokens/s.
- Swapping the attention implementation to FlashAttention-4 adds another 11% improvement.
- Combined, overall throughput increased by 15% without altering the architecture or retraining.
- The author notes there is still significant room for optimization in the MoE layers.
The core takeaway is that the kernels approach allows model developers to quickly integrate better implementations just like changing a config, while kernel developers focus on delivering the fastest versions, ultimately benefiting everyone.
More from Infra
- ClickHouse aggregates 100B rows in 0.4s with parallel replicas auto-scaling — Abhishekcur · 2026-10-18
- Auction-based LLM routing paper finds fixed token pricing wastes 10%-71% in costs — _onionesque · 2026-10-18
- Running Ling 3.1 Flash (560B MoE, 25B Active) Across 4 DGX Sparks via llama.cpp RPC — BerryBlush_Rogers691 · 2026-10-18
- SGLang Summit sponsors jump from 11 to 36 in one week — ying11231 · 2026-10-18
- Oracle CDS hits record 300bps, called first AI trade domino after Barclays sell downgrade — arthurcolle · 2026-10-18
- Agent Coding Workflow Would Cost ~$40k/Month on API Pricing, Dev Reveals — ChrisRackauckas · 2026-10-18