Inkling-2 Mini hits 210 tok/s on Mac Studio, builds apps 1.8x faster than Sonnet
SonglinYang4 · x · 2026-11-21
ExoLabs partnered with Thinking Machines to heavily optimize Inkling 2 Mini for local inference on Apple silicon: 210 tok/s decode and 3620 tok/s prefill on a 96GB M5 Ultra Mac Studio, and 125 tok/s decode / 1750 tok/s prefill on a 48GB M5 Max MacBook Pro.
In a test it built a native weather app 1.8x faster than Claude Sonnet 5.5 on M5 Ultra, and 11x cheaper than API pricing, with claimed Sonnet-level performance. The key is its looping architecture: reusing layers boosts inference-time compute without growing the model, ideal for memory-constrained local devices.
More from coding & agent
- rasbt pushes back on MIT CSAIL paper claiming harnesses won't be obsolete — rasbt · 2026-11-22
- Gemini Neon builds an interactive USL lock-contention UI exposing a testing blind spot — tianyin_xu · 2026-11-22
- Claude Code can now be approved from your iPhone, ending the 14-minute permission wait — BLCNYY · 2026-11-22
- AMD and PyTorch ship a playbook to run GPT-OSS-20B inference locally with ROCm — PyTorch · 2026-11-22
- Keep context out of the prompt: RLM-27B beats base Qwen3-27B by 26.9% — dosco · 2026-11-22
- instant_record compiles Rails ActiveRecord models to WASM for instant, offline browser apps — danshipper · 2026-11-22