Five inference engines on M3 Max 36GB: Kapitsa hits 41 tok/s dense, oMLX ties on MoE
Hyungsun · reddit · 2026-10-24
The author ran a second systematic comparison of five Apple Silicon inference engines on an M3 Max (36GB), adding Kapitsa 0.2.12 and Flint 40B-A2.5B, and switching oMLX to the oQ4e-mtp + Lightning MTP checkpoint as requested.
Key results (median of 3, sustained 256-token prose)
- Dense Qwen3.8-27B-4bit: Kapitsa 41.3 tok/s fastest; oMLX 34.1, Splash 33.0, MTPLX 31.6, Rapid-MLX 26.2.
- MoE Qwen3.6-35B-A3B: oMLX 122.9 and Kapitsa 122.7 tie at the top; Rapid 110.3, Splash 97.0, MTPLX 79.9.
- Flint 40B-A2.5B: Kapitsa hits 134.2 tok/s; runs on 36GB but not the daily default.
Takeaways
- oMLX's dense jump from 17.9 to 34.1 mostly comes from speculative decoding checkpoints (75–85% draft acceptance), not a free engine-only win.
- Rapid-MLX 0.16.8 slightly regressed on single-stream; its focus moved to Flint and concurrency.
- MTPLX's near-identical scores across runs validate the test protocol.
- Weight builds differ per engine, so read results as engine+build.
The author's new daily default is Kapitsa 0.2.12 with the 35B-A3B model.
More from Infra
- Hunyuan 3D 2.1 nearly unusable on Mac: errors above 512 resolution — idmimagineering · 2026-10-24
- GPU programming day 293: diving into SGLang via mini-sglang and MIT's EfficientML lecture 12 — NandoDF · 2026-10-24
- China's Jiuzhang 4.0 claims 10^42-year speedup in 25 microseconds — but the task has no practical use — AryHHAry · 2026-10-24
- Fancy coding-agent harnesses cost 3x more but don't improve pass rates, study finds — The Decoder · 2026-10-24
- New Gyro quant format accused of being a closed-source EXL3 clone with no proven gains — nasone32 · 2026-10-24
- Cutting KServe cold starts from 134s to 9s with warmctl — kbesplv · 2026-10-24