MLX-Serve adds Inkling 2 Mini with ~24 custom kernels, fastest Mac inference yet
SumitGup · x · 2026-11-23
ddalcu announced MLX-Serve support for Inkling 2 Mini, claiming 24 custom kernels make it possibly the fastest way to run the model on a Mac today. Quantization support covers 8-bit plus NVFP4 and MXFP8; 4-bit hasn't passed their quality bar yet. ivanfioravanti is separately optimizing for M3 Ultra.
More from Infra
- Open-source Kapitsa hits 960+ tok/s prefill on M5 Max with 4-bit Hy4-flash — 0zyman23 · 2026-11-23
- Alabama Power to cut rates $160M thanks to data center revenue — binarybits · 2026-11-23
- One-line MLX fix makes long-context inference up to 10.1x faster on M5 Max — SumitGup · 2026-11-23
- Anthropic's $240B cash pile will go to semiconductors as RSI models demand compute — firstadopter · 2026-11-23
- Fine-tuned Inkling 2 Mini routes tickets with early exit: median latency 136.8ms to 81.4ms at 0.07% accuracy cost — eltokh7 · 2026-11-23
- TechCrunch talks to Paris Marx on "Hyperscale" and the data center backlash — TechCrunch AI · 2026-11-23