Flint 40B hits 128 tok/s on M4 Pro via Kapitsa, 1.9x faster than llama.cpp
DotaMate · reddit · 2026-10-22
Kapitsa (github.com/nace-ai/kapitsa), an open-source Apple Silicon inference engine with custom Metal kernels, benchmarked Recursive's newly released coding model Flint 40B A2.5B Q4 at 128 tok/s decode on a MacBook Pro M4 Pro 48GB. Kapitsa exposes OpenAI/Anthropic-compatible APIs and works with Claude/Codex/Pi/OpenCode.
Benchmark vs Recursive's own rec 0.1.0 and llama.cpp b11804:
- Short-context decode ties rec (within 1%); both 1.6x faster than llama.cpp
- Lead grows with context: 1.30x over rec and 1.9x over llama.cpp at 128K decode; prefill 1.1-1.2x rec at all lengths
- Cached second-turn time-to-first-token: 0.13s vs rec's 0.41s (3x faster)
- Lowest peak memory: 23.3GB
The team also trained a KFlash tree drafter for speculative decoding on an 8xH200 rack, giving 1.4x decode speedup internally, releasing Monday. Requires a 32GB+ Mac; 22.6GB download. Known limits: Flint always reasons first, prefill on M5+ not yet optimized. A kapitsa bench command publishes benchmark PRs to crowdsource more devices.
More from Infra
- TSMC surveys suppliers on rare-earth-free motors for factory automation — pstAsiatech · 2026-10-22
- Zoho cries poor over AI costs, then applies to build $200M semiconductor fab — kingslyj · 2026-10-22
- NVIDIA recaps GTC Berlin dev community: open models to physical AI — NVIDIAAI · 2026-10-22
- GPU payback math: B300 rigs recoup costs in 10 months, 1GW build could earn ~$38B a year — tctjr · 2026-10-22
- What do customers actually get for billions in AI capacity prepayments? 20 companies reviewed — tengyanAI · 2026-10-22
- Kokon 0.10 reuses Qwen-Image 2.1's text encoder to write prompts, runs on 8GB cards — Ok-Director4174 · 2026-10-22