oMLX 0.8 turns NVIDIA GPUs and DGX Spark into prefill accelerators for Mac, up to 5.3x faster
pcuenq · x · 2026-11-16
oMLX 0.8.0rc1 adds a Prefill Accelerator: an NVIDIA GPU PC, DGX Spark, or second Mac runs the first layers of prompt prefill while your Mac keeps decoding with the KV cache. No RDMA or Thunderbolt needed — just LAN, and one docker run on the GPU side. On a 64K prompt with M3 Ultra 512GB: Qwen3.8 27B jumps from 352 to 1,062 tok/s with DGX Spark (+202%) and to 1,879 tok/s with 2x RTX A6000 (+434%). Auto mode splits layers based on measured speeds.
More from Infra
- CBS reports on China's massive AI data center boom in Ulanqab, Inner Mongolia — armano · 2026-11-16
- China's AI Edge Is the Full Stack: Open Models, Humanoids, Energy, Materials — ingliguori · 2026-11-16
- LateTalk guest: AI infrastructure buildout may already be done, apps are still early — oran_ge · 2026-11-16
- Export controls target China's EUV self-sufficiency, not GPU smuggling, analyst argues — teortaxesTex · 2026-11-16
- Cyberpunk 2077 gets a stunning visual overhaul with DLSS 5 Double Pass and Path Tracing — Promptmethus · 2026-11-16
- MMFlash boosts vision-language speculative decoding with up to 3.58x lossless speedup — _akhaliq · 2026-11-16