TurboEXL3 Hits 140 TPS on M5 Macs With EXL3 Quantization
HankYeomans · x · 2026-10-29
- Developer David Tai released TurboEXL3, a fused-kernel library that makes the EXL3 quantization format one of the fastest on M5/NAX-enabled Macs.
- Benchmarked on Qwen 3.8 Flash: up to 140 TPS decode with a full EXL3 conversion, with prefill performance on par.
- Speedup comes from fusing encode/decode kernels directly into batched GEMM/GEMV operations; EXL3's accuracy gains boost token drafting, offsetting quantization losses.
- Practical picks: EXL3 @ 5.05 bits is near-lossless vs BF16; @ 3.05 bits matches NVFP4 accuracy at 33% smaller size.
- Coming soon to mlx-serve, with a full technical write-up.
More from Infra
- Why Macs lose to DGX Spark at batched inference: bandwidth wins solo, FP4 compute wins crowds — QuixiAI · 2026-10-29
- ComfyUI reproducibility: the workflow JSON alone isn't enough — Im_bored_rn69 · 2026-10-29
- Akamai launches Akamai Inference and Agent Sandbox for enterprise AI deployment — lucaolivari · 2026-10-29
- 512GB M5 Ultra Priced at $22,300, Earliest Pickup December 3 — rez0__ · 2026-10-29
- Lumentum CEO says copper-to-optics upgrades will drive growth even if AI buildout cools — pstAsiatech · 2026-10-29
- Training speedrun record smashed: 30M model in 22.9s, 21.8x faster — RobertTLange · 2026-10-29