Inkling 2 Mini runs at 125 tok/s on a MacBook, claims best-in-class at its size
simonguozirui · x · 2026-11-21
exolabs partnered with Thinking Machines to make the open-weight Inkling 2 Mini run fast locally: 210 tok/s decode and 3620 tok/s prefill on a 96GB M5 Ultra Mac Studio, and 125 tok/s decode with 1750 tok/s prefill on a 48GB M5 Max MacBook Pro.
In testing it built a native weather app 1.8x faster than Claude Sonnet 5.5 and 11x cheaper on API pricing. The author calls it an American open-weight model competitive with Qwen 27B, likely the best at this size.
The looped architecture suits local devices: looping layers adds inference-time compute (and intelligence) without growing the model, ideal for memory-constrained MacBooks.
More from Infra
- How do you run big batch GPU jobs — thousands of files, frames, or prompts? — mrtonymat · 2026-11-21
- ComfyUI model loading stuck at 400 MB/s even after upgrading to NVMe — C-h_A_o-S · 2026-11-21
- Inkling 2 Mini hits ~200 tok/s on a single RTX 4090 with FP8 + NVFP4 quantization — eesahe · 2026-11-21
- Same RTX 5090, 4x throughput: Surogate hits 2,765 tok/s vs llama.cpp's 683 with 8 users — AIFlow_ML · 2026-11-21
- Compound's AI slowdown scenario: NVDA down ~45% if revenue growth halves and multiples revert — tengyanAI · 2026-11-21
- DatologyAI open-sources Zephon, a data loader with elastic determinism for LLM training — josh_wills · 2026-11-21