Open-source Inco Splash hits 144 tok/s running Qwen3.8-27B on M5 Max, 3x faster than Ollama
aigclink · x · 2026-09-20
Open-source inference engine Inco Splash claims 144 tok/s decoding Qwen3.8-27B (4-bit) on an M5 Max MacBook Pro — up to 3x faster than Ollama and 2x vs oMLX, and near 4x in aggregated throughput for 4 concurrent sub-agents with short prompts.
- Currently supports Qwen3.8-27B and Qwen3.6-35B-A3B on Apple silicon only
- Approach: kernels, draft model, and memory plan are specialized per model, with zero configuration
- Author's take: if these small models get fast enough on Macs, some coding workloads will shift back from cloud APIs to local inference
More from coding & agent
- Veteran game dev: "AI builds a full game in a day" posts are mostly hype — bendee983 · 2026-11-04
- Dev builds interactive Excel-to-dashboard with ChatGPT, shares the prompt — TermRemarkable665 · 2026-11-04
- Agents cut dev time from weeks to a day; bottleneck moved to specs and review — Suspicious_Orchid770 · 2026-11-04
- Orgo, a YC-backed platform giving AI agents their own cloud desktops, claims 1,000+ teams — Scobleizer · 2026-11-04
- Founder: Vibe-coded a PMF-successful product, then hired pro devs to fix the mess — sujingshen · 2026-11-04
- One prompt to audit cross-region traffic: dev cut trans-Atlantic latency by moving a service — DanielLockyer · 2026-11-04