Open-Source mlx-dspark Boosts LLM Inference on Mac by 3.3x
A-Rahim · reddit · 2026-08-13
Developer A-Rahim released the open-source project mlx-dspark, which leverages speculative decoding on Apple Silicon to significantly accelerate Meta's Muse Glimmer 30B model.
In real-world testing on an M4 Pro chip, the 8-bit model's generation speed increased from 8.2 tok/s to 18-26 tok/s. Math tasks saw the best improvement (3.27x), followed by code (2.5x) and chat (2.22x). Because the target model verifies every token, the output is byte-identical to normal decoding, meaning there is no quality trade-off.
Additionally, the 4-bit quantized run achieves 25 tok/s (1.7x speedup) requiring only 18GB of memory, while the 8-bit run peaks at 40GB. This effectively delivers 8-bit quality at 4-bit speeds.
More from Infra
- New Quantization Framework to Run 1.6T Models on a Single B300 GPU at 50 tok/s — dosco · 2026-08-13
- Menlo Park Daytime Electricity Hits 53.8¢/kWh: Running Own GPUs Becomes Irrational — generativist · 2026-08-13
- Running Krea 2 Turbo on 8GB VRAM: RTX 3070 Ti Local Test — niechta · 2026-08-13
- Fluidstack Visits NYSE to Discuss US AI Infrastructure Investment — MxMnr · 2026-08-13
- Open-Source CUDA Alternative for Portable AMD GPU Code — tom_doerr · 2026-08-13
- Slow TPS on AMD 7900 XT Running Gemma: 131K Context Bottleneck — opoot_ · 2026-08-13