M5 Max benchmark: Kapitsa hits 36.6 tok/s with KFlash, 28GB at 32K vs mlx-lm's 52GB
DotaMate · reddit · 2026-10-27
A developer (author of open-source Apple Silicon engine Kapitsa, self-declared conflict of interest with repro scripts) benchmarked Nanbeige4.3-33B 4-bit on a MacBook Pro M5 Max 128GB across mlx-lm, llama.cpp, and Kapitsa from 512 to 32K tokens. Key results: mlx-lm leads prefill (546 tok/s at 32K); Kapitsa with KFlash speculative decoding hits 36.6 tok/s writing code vs 27.1/24.9; follow-up-turn TTFT is 140ms vs 500ms; peak memory at 29.5K is 28GB vs 52GB. The model's per-layer full attention makes its KV cache very inefficient (26GB for 128K tokens).
More from Infra
- WireGuard tutorial: private encrypted tunnels between devices on any network — victorialslocum · 2026-10-27
- DeepSeek-V4.1-Flash 1.75bpw quant released for 128GB Macs — bobby-chan · 2026-10-27
- Unsloth details running DeepSeek-V4.1-Flash locally: 1-bit GGUF fits 128GB RAM with 1M context — gnukeith · 2026-10-27
- SK hynix down 47% since July while DDR5 spot prices rose 31%: a striking disconnect — tengyanAI · 2026-10-27
- JustGOAL paper: general frontier model beats specialized GPU kernel agents — azed_ai · 2026-10-27
- NVIDIA's Nemotron-3.5 Bolt 74B-A9B matches Super within 1% with 9B active params — ryanshrout · 2026-10-27