Thinking Machines releases Inkling 2 Small, runnable on 4 GPUs with open weights coming
Thinking Machines has released a new model, Inkling 2 Small, which it officially describes as an open weight model. It already supports fine-tuning and inference on Tinker, with full weights to be open-sourced soon. The model scores 31 on the AA Intelligence Index, a big jump from its predecessor; the third-party team Specific tested it on the Real-SWE 1.1 benchmark, where the solve rate quadrupled from 8% (Inkling 1 Small) to 32%.
Confirmed
- Official release: scores 31 on the AA Intelligence Index, available for fine-tuning and inference on Tinker, full weights to be open-sourced soon
- Architecture: 594B parameters, MoE design with 8 shared router experts and 48 router experts, only 48 layers (vs. 78 for Hy4)
- Attention Residuals span 8 blocks; Sebastian Raschka hand-drew an architecture diagram and commented that it's a more interesting model than he expected
- Deployment cost: after EXL3 3.48bpw quantization it fits on 4×RTX Pro 6000 (96GB Blackwell) or 3×DGX Spark, with KV cache capacity up to 7.5 million tokens (a repost claims 380K concurrent KV cache on 4 GPUs)
- Third-party evaluation: Specific measured a 32% solve rate on Real-SWE 1.1 (8% for the predecessor)
Why it matters
The model delivers flagship-level capability at a relatively low hardware bar (4 consumer/professional GPUs), and with weights soon to be open-sourced plus aggressive architectural innovations (Attention Residuals, fewer layers), it holds strong appeal for the local inference and fine-tuning ecosystem. liuliu notes that although the model is already downloadable, discussion has been muted; third-party benchmark data may change that.
2026-11-14 ~ 2026-11-16 · 5 related posts
- Episode 1: Thinking Machines Open-Sources Inkling 2 Small Weights Under Apache 2.0(2026-11-13, 31 posts)
- Episode 2: vLLM adds Thinking Machines' Inkling-2-Small with 1M context and 1100 tok/s(2026-11-13, 2 posts)
- Episode 3: Thinking Machines Open-Sources Inkling 2 Small, Top US Open Model on Arena(2026-11-13, 3 posts)
- Episode 4: Inkling 2 Small Tops Eval³ Leaderboard for Open-Weight Models(2026-11-13, 2 posts)
- Episode 5: Thinking Machines' Inkling 2 Small lands on Ollama with 1M context(2026-11-13, 2 posts)
- Episode 6: Thinking Machines' Open-Weights Inkling Small 2 Wins Praise(2026-11-13, 2 posts)
- Episode 7: Unsloth Releases Inkling-2-Small GGUF Quants, Running 594B Model Locally in 171GB(2026-11-13, 2 posts)
- Episode 8: Thinking Machines Reveals End-to-End 4-Bit Training for Inkling 2 Small(2026-11-14, 10 posts)
- Episode 9: Researchers Question Payoff of Inkling 2 Small's Native 4-bit Training(2026-11-14, 2 posts)
- Episode 10: Thinking Machines releases Inkling 2 Small, runnable on 4 GPUs with open weights coming(2026-11-14, 5 posts)
Primary sources
- [source] Third-party test: Thinking Machines' Inkling 2 Small jumps to 32% on Real-SWE, 4x its predecessor — keirp1 · 2026-11-14
- Inkling Small 2 fits on 4x RTX Pro 6000 with 7.5M token KV cache at 120 tok/s — LiTianleli · 2026-11-14
- Inkling 2 Small quietly ships: 48-layer model with attention residuals and new sinks — PMinervini · 2026-11-14
- [source] rasbt maps Inkling 2 Small's 594B architecture built on attention residuals — rasbt · 2026-11-14
- Thinking Machines releases Inkling 2 Small; 4-GPU setup hits 380K KV cache concurrency — LiTianleli · 2026-11-16