FULL STORY
Kapitsa: A Handwritten Inference Engine Speeds Up the Mac
A developer released Kapitsa, a fully handwritten Apple Silicon inference engine using tree speculative decoding. Version 0.2.26 later hit 346 tok/s on Qwen 35B on an M5 Max.
2026-10-24 ~ 2026-11-02 · 2 episodes · 4 posts
Episode 1 · Open-source engine kapitsa boosts Mac local LLM inference with tree-based speculative decoding (2026-10-24, 2 posts)
A developer released kapitsa, a hand-written C++20 + Metal inference engine for Apple Silicon featuring tree-based speculative decoding (kflash). It runs Qwen3.8-27B at 140 tok/s on Mac, twice as fast as Ollama, with 23% higher multi-agent throughput.
Episode 2 · Kapitsa 0.2.26 Benchmarks: Qwen 35B Hits 346 tok/s on M5 Max (2026-11-02, 2 posts)
Open-source Apple Silicon inference engine Kapitsa released version 0.2.26, with benchmarks on the M5 Max (128GB) showing Qwen 35B at 346 tok/s and a 180B model running at 61 tok/s on a 48GB Mac via SSD offloading.
- Kapitsa hits 346 tok/s on a 35B MoE and runs a 180B model on 48GB Macs via SSD offloading — DotaMate · 2026-11-02
- Kapitsa inference engine hits 346 tok/s on Qwen 35B, runs 180B via SSD offload on M5 Max — DotaMate · 2026-11-02