Inkling 2 Mini hits 210 tok/s locally on M5 Ultra, builds app 1.8x faster and 11x cheaper than Claude
pcuenq · x · 2026-11-22
ExoLabs partnered with Thinking Machines to run Inkling 2 Mini fast on local Macs:
- 210 tok/s decode, 3620 tok/s prefill on a 96GB M5 Ultra Mac Studio
- 125 tok/s decode, 1750 tok/s prefill on a 48GB M5 Max MacBook Pro
In a real test it built a native weather app 1.8x faster than Claude Sonnet 5.5 via API, at 11x lower cost.
The key is the looping architecture: cycling through model layers adds inference-time compute (and intelligence) without growing the model, which suits memory-constrained devices. An HF staffer called it a sign that local, open-source AI is how the underdog wins.
More from Infra
- Huawei Ascend-950DT quoted at 3x 950PR price; yield and SuperPod interconnect may explain the gap — teortaxesTex · 2026-11-22
- AlphaChip and three TPU generations: a mine that digs itself is a throne — krishnan · 2026-11-22
- SK Hynix reportedly tops 70% yield on HBM4, starting mass production in Q3 — Beth_Kindig · 2026-11-22
- Mflux Runs MOSS Image Gen on 8GB Macs: 82s for 768x768 With 4-Step Distill LoRA — bio_risk · 2026-11-22
- After a Possible Riemann Proof, a Call to Reinvent AI Memory and Inference Cost — Kitchen_Mirror_7247 · 2026-11-22
- Intel D270 AI card: 160GB LPDDR5X, 350W, ~$10K, and why call it a GPU? — Terminator857 · 2026-11-22