MLX beats llama.cpp 2x on M5 Pro for Inkling Mini, NVFP4 fallback to blame

xlr8harder · x · 2026-11-22

In a thread on the fastest local inference for inkling mini, muradylan shares Apple Silicon numbers: MLX hits 37 tok/s vs 17.7 tok/s for llama.cpp on an M5 Pro — about 2x faster — because the GGUF's NVFP4 layers fall back to CPU on Metal. For bulk jobs, keep reasoning effort low, since high thinks for ages.

Related event: Inkling 2 Mini Runs Smoothly on 24GB M5 Pro, MLX Beats llama.cpp(2 posts)→

Original post →

More from Infra

Infra channel →