Kapitsa 0.2's KFlash tree speculative decoding beats Splash 1.4.0 with 63 tok/s on Qwen 27B
DotaMate · reddit · 2026-10-21
Kapitsa, an open-source Apple Silicon-optimized inference engine for Qwen and Nemotron models, shipped v0.2 with a new speculative decoding method called KFlash, benchmarked against Inco's Splash 1.4.0, Ollama, llama.cpp, LM Studio and more on consumer Macs.
How KFlash works: most fast engines use a small draft model guessing 7 tokens that the big model verifies in one pass. DFlash2 checks a single chain and stops at the first wrong guess; KFlash instead verifies a best-first tree of the drafter's candidates in the same pass, so runner-up guesses get checked when the drafter hesitates.
Results (512-token coding answer decode speed, 4-bit, each engine at fastest config):
- MacBook Pro M4 Pro 48GB: Kapitsa leads across the board — 63 tok/s on Qwen3.8 27B (vs Splash's 53), 161 tok/s on Qwen3.6 35B-A3B, and 201 tok/s on Qwen3.5 4B, roughly 3x llama.cpp
- MacBook Pro M1 Pro 16GB: 95 tok/s on Qwen3.5 4B, while Splash requires M3+ and can't run the small model
Kapitsa supports all M1–M6 chips, serves OpenAI/Anthropic-compatible APIs, hooks into OpenCode/Claude-style coding tools via CLI, and kapitsa bench lets anyone reproduce and submit community benchmark results.
More from coding & agent
- Community builds 4 GUIs for the Pi coding agent as official client stalls — solyarisoftware · 2026-10-21
- Nace.AI Launches Drex Max 2.0, a 27B Local Decision Model for Computer Use — DotaMate · 2026-10-21
- Musk shares how Grok Bot cleared 204 support tickets in ~10 minutes with one engineer approval — elonmusk · 2026-10-21
- pstack ships meta-skill that has your agent draw statecharts, then find bugs via TLA+ formal verification — DavidKPiano · 2026-10-21
- Study: decision-model controllers raise agent cost 19% with zero success gain — Naman Vats · 2026-10-21
- Tencent's RiseHarness iteratively evolves agent harnesses without touching model weights — Tencent-Hunyuan · 2026-10-21