Thinking Machines Launches Inkling-Small: 276B Params, Only 12B Active
simonguozirui · x · 2026-07-31
Thinking Machines Lab has officially released the Inkling-Small model. It is a natively multimodal architecture (supporting text, image, and audio inputs) with a total of 276B parameters, but only activates 12B parameters per token.
- Performance & Cost: At a quarter of the size of the original Inkling, it matches its capabilities and even wins on some benchmarks, significantly lowering deployment costs.
- Inference Optimization: Using SGLang on 8x NVIDIA B200, it achieves 648 tok/s decode speed with DSpark enabled, and 288 tok/s without.
- Fine-tuning: The parameter size is a sweet spot for RL. Both LoRA and full-parameter training are well within reach, and the Miles platform is verified for multimodal RL.
- Deployment: Supported by vLLM, it requires a minimum of 180GB VRAM (NVFP4 format) to run, supporting up to 1M context length.
Related event: Inkling-Small: 276B params, 12B active, open-source tops ARC(48 posts)→
More from Infra
- SkyRL ships Day-0 RL support for Beam with +62% LoRA decode throughput in vLLM — vllm_project · 2026-10-16
- DFlash2 delivers up to 3.03x decode-only speedup for Beam across five benchmarks — vllm_project · 2026-10-16
- vLLM's Beam serving kernels cut GB200 forward time from 8.15 to 6.47 ms at TP8 batch 128 — vllm_project · 2026-10-16
- Reflection's 501B MoE Beam hits 463 tok/s on 8x GB200 with NVFP4 at day 0 — zhyncs42 · 2026-10-16
- Nemotron 3.5 Bolt runs 16 concurrent streams on a single DGX Spark — NVIDIAAI · 2026-10-16
- Why your computer can't store 0.2: an IEEE 754 primer — Franc0Fernand0 · 2026-10-16