Open-source Kapitsa: a Metal Engine for Apple Silicon Pushing Qwen 27B to 63 tok/s on M4 Pro
DotaMate · reddit · 2026-10-19
Nace.ai open-sourced Kapitsa (Apache 2.0), a local inference engine for Apple Silicon built to run coding agents on models that fit in Macs—aimed at enterprise networks with no internet access. It supports only Qwen and NVIDIA Nemotron families so Metal kernels can be customized per architecture.
Benchmarks: on M4 Pro 48GB, Qwen3.8 27B hits 63 tok/s (vs 32 on Ollama) and Qwen3.6 35B-A3B hits 154 tok/s. On M5 Max 128GB, 35B-A3B reaches 272 tok/s and Nemotron Bolt 74B-A9B reaches 127 tok/s (unsupported by most engines). The one loss: Flash-Next 180B-A7B, where mlx-serve leads by 2-3%.
Where speed comes from:
- Custom Metal kernels per architecture (Gated DeltaNet + attention for Qwen3.5, Mamba-2 + LatentMoE for Nemotron-H) and per weight format (MLX affine, NVFP4, 2-bit experts)
- Speculative decoding: DFlash2 for Qwen 27B/35B, z-lab's DFlash for 4B, DSpark for Lightning, NVIDIA's LiLiCorr for Bolt/Super-VL
- Hybrid-model caching: recurrent layers break plain KV prefix caches, so the engine keeps KV plus recurrent-state checkpoints at turn boundaries and preloads your last answer while you read it
Install via brew install nace-ai/tap/kapitsa; run your own benchmarks with kapitsa bench. Contributors welcome.
More from coding & agent
- BELIEFWEAVER: belief updates and world modeling for long-horizon agent failure recovery — IntelligenceLab · 2026-10-19
- DeepMind's XYEval: one bad user hint drops agent success from 67.4% to 36% — i_dg23 · 2026-10-19
- REA: open-source MCP tool lets AI agents reverse engineer any app down to assembly — Aiden_Tech_Ai · 2026-10-19
- Pure three.js + Claude/Codex Yields Fully Procedural Game Characters — TinfoilTricorn · 2026-10-19
- One CLI Beats One MCP Tool per API: an Agent Tooling Workflow and a 7-Day OAuth Trap — tariqosmani · 2026-10-19
- vLLM's agentic-api: stateful continuation ends 200K-token resends for agent harnesses — AccBalanced · 2026-10-19