Kapitsa engine hits 97 tok/s on Qwen3.5 4B in benchmarks on a 2021 M1 Pro MacBook with 16GB
DotaMate · reddit · 2026-10-22
The Kapitsa project (v0.2.7), an open-source inference engine with custom Metal kernels optimized per Apple Silicon chip, published systematic 1K–32K context benchmarks on a 2021 MacBook Pro M1 Pro (16GB) against mlx-serve, mlx-lm and llama.cpp.
Key results (decode, median of 5):
- Qwen3.5 4B: 97.2 tok/s, 1.56× mlx-serve, 4.4GB peak RAM
- Qwen3.5 9B: 56.8 tok/s (1.52×)
- Qwen3.6 35B-A3B 2-bit: 52.2 tok/s, 1.21–1.35× llama.cpp with far less swapping (36K vs 207K pages)
- Nemotron 3 Nano 4B: 1.31× decode, 1.45–1.67× prefill vs llama.cpp
- Nemotron Dory 4B leads at short context but falls behind mlx-lm past 8K
Details: KFlash tree speculative decoding works on M1; prefill lead grows with prompt length (31.5K prompt: first token in 101s vs 154s on mlx-serve). Install via brew install nace-ai/tap/kapitsa; serves OpenAI/Anthropic-compatible APIs locally.
More from Infra
- SGLang adds live GPU scaling for MoE deployments without restarts via Elastic EP + NIXL — ying11231 · 2026-10-22
- US announces $389M investment in advanced chips for fusion, AI data centers — Polymarket · 2026-10-22
- Qdrant Facet API: building accurate search sidebar counts — qdrant_engine · 2026-10-22
- Kangwook Lee details inference setup: W4A16 quant, ~120 tok/s single-stream without drafter — Kangwook_Lee · 2026-10-22
- Neoclouds hit $9B quarterly revenue, up 223%, but can't sell to the $400B enterprise market — DavidLinthicum · 2026-10-22
- capy-flash v0.7 runs a 125B MoE in 64GB RAM, hitting 234 tok/s on two 16GB cards — FeydRowan · 2026-10-22