oMLX lets a second Mac boost prefill up to 2.9x on big-memory Macs
pcuenq · x · 2026-11-05
oMLX 0.7.4.dev1 introduces a Prefill Accelerator: pair a fast-GPU Mac (M5 Max) with a big-memory Mac (M3 Ultra 512GB) over Thunderbolt or 5GbE — no RDMA needed. The fast Mac runs the first layers of each prompt chunk while the big Mac handles the rest, decoding with unchanged speed.
64K prompt results (M3 Ultra 512GB + M5 Max 128GB):
- DeepSeek V4.1 Flash: 1,060 → 3,123 tok/s (+195%)
- Qwen3.8 Flash-Next: 1,309 → 3,666 tok/s (+180%)
- GLM-5.3 Flash: 540 → 1,388 tok/s (+157%)
- MiniMax M3.1: 297 → 569 tok/s (+92%)
- An M5 Max 64GB still adds 30-118%
Also includes A8 and buffer pool improvements for DeepSeek V4.1 (43-48% faster prefill on M5 Ultra) and fused FP16 kernels for Qwen3.8 Flash-Next (decode 38.9 → 63.0 tok/s on M3 Ultra).
Related event: oMLX update lets a second Mac accelerate prefill(2 posts)→
More from Infra
- Data centers are hated for fixable reasons — noise, looks, water, energy — yet no one tries — dansitu · 2026-11-05
- 128GB RAM plus dual 3090s: is upgrading a home AI server still worth it? — LankyGuitar6528 · 2026-11-05
- SpaceX plans 747-acre Arizona data center campus with up to 6GW power demand — XFreeze · 2026-11-05
- Is China's AI video boom eating the compute it needs for LLM training? — enilea · 2026-11-05
- Broadcom's VCF update targets private AI with model sharing, confidential computing — DavidLinthicum · 2026-11-05
- Traffic down 50% but Supabase bill up 300% and Vercel up 400% — DanielLockyer · 2026-11-05