Inkling 2 Mini runs at 125 tok/s on a MacBook, claims best-in-class at its size

simonguozirui · x · 2026-11-21

exolabs partnered with Thinking Machines to make the open-weight Inkling 2 Mini run fast locally: 210 tok/s decode and 3620 tok/s prefill on a 96GB M5 Ultra Mac Studio, and 125 tok/s decode with 1750 tok/s prefill on a 48GB M5 Max MacBook Pro.

In testing it built a native weather app 1.8x faster than Claude Sonnet 5.5 and 11x cheaper on API pricing. The author calls it an American open-weight model competitive with Qwen 27B, likely the best at this size.

The looped architecture suits local devices: looping layers adds inference-time compute (and intelligence) without growing the model, ideal for memory-constrained MacBooks.

Original post →

More from Infra

Infra channel →