Inkling Performance and Related Architecture Speculation
@nrehiew_ posted a rapid series of analyses on July 16 around the newly released Inkling model, starting with benchmark behavior and product positioning, then extending into architecture and training-signal discussion around a related frontier-style design. The thread drew interest for two reasons: why a model said to be about 3.5x smaller can still post unusually strong results, and whether very large MoE systems are making new trade-offs in attention, positional encoding, and training setup.
Inkling performance and positioning
In @nrehiew_’s reading, Inkling is aimed more at general-purpose reasoning than at being a pure coding agent, which he said fits its dependence on the Tinker platform and a strategy he compared to Cohere. He also pointed to tasks such as TerminalBench and SimpleQA as useful lenses for seeing what parts of model capability are doing the work in these results.
Architecture details under discussion
In a separate breakdown of a suspected frontier model design, @nrehiew_ said the attention stack uses a sliding window and basic 1D convolutions on KV and residual paths. He also highlighted the absence of RoPE, replaced instead by relative position bias collected from distance information. On the MoE side, he summarized figures of about 975B total parameters, 41B active parameters, and roughly 45T multimodal training tokens, adding that both active parameter count and token scale appear larger than Dsv3.
What remains unconfirmed
@nrehiew_ further speculated that a mechanism described as adjusting cost by token might effectively be changing the length penalty according to “thinking intensity,” which could help explain a more compressed CoT style. He explicitly framed this as inference from observed results rather than an official confirmation. A repost shared by @burny_tech also relayed community-side views that muP may matter again at the 1T+ scale, and that relatively low sparsity could reflect hardware constraints or lower expert parallelism; those points likewise were not official disclosures.
2026-07-16 ~ 2026-07-16 · 7 related posts
- Episode 1: Thinking Machines 首发开权重 Inkling(2026-07-16, 169 posts)
- Episode 2: Inkling 表现与相关架构猜测升温(2026-07-16, 7 posts)
- Episode 3: 新开源模型 Inkling 实测表现翻车(2026-07-16, 2 posts)
- Thinky Model Scaling and Multimodal Insights — burny_tech · 2026-07-16
- Further Details on MoE Configuration — nrehiew_ · 2026-07-16
- [source] Suspected Frontier Model Architecture: No RoPE and Hybrid Attention — nrehiew_ · 2026-07-16
- Deep Dive into a Model's Attention and MoE Details — nrehiew_ · 2026-07-16
- Adjusting Cost by Reasoning Intensity? — nrehiew_ · 2026-07-16
- [source] Can Small Models Be Incredibly Strong Too? — nrehiew_ · 2026-07-16
- [source] Analyzing Inkling: General Reasoning in an Ultra-Compact Size — nrehiew_ · 2026-07-16