Inkling 2 Mini: a 26B looped MoE that runs agentic workloads locally at 168 tok/s
ying11231 · x · 2026-11-21
ThinkyMachines released Inkling 2 Mini, answering community feedback that Inkling 2 Small (594B) was too big.
- Architecture: a 26B MoE (5.4B active) with a looped design — middle layers run 3 times, keeping the depth of a much larger model at laptop-runnable size
- Capability: scores 20.1 on the AA Index, described as sufficient for agentic work
- Local inference optimizations (SGLang, AgentX concurrency 1, p90): 168 tok/s on RTX Pro 6000 / RTX 5090, 24 tok/s on DGX Spark, with native multi-GPU inference
The SGLang team boosted the release, noting they put significant work into optimizing it for local devices.
More from Infra
- Microsoft's USB4 RDMA lets mini PCs pool memory to run Qwen3-235B — JFPuget · 2026-11-22
- LTX 2.5 keeps crashing on AMD 9070 XT with driver timeouts while MiniMax H3 runs fine — ZestycloseReading504 · 2026-11-22
- FT: AI server suppliers raise prices up to 45% months after deals are signed — nordicinst · 2026-11-22
- China Installed 59% of New Industrial Robots in 2026 Report, ~354,000 Units — ingliguori · 2026-11-22
- MLX over Vulkan hits 124 tok/s on RTX 5060 Ti, beating llama.cpp CUDA by 25% — SumitGup · 2026-11-22
- Intruder swapped Prime Intellect's hosted DeepSeek for an abliterated copy for 53 hours — mawcopolow · 2026-11-22