mlx-serve PR runs DeepSeek-V4.1 tensor-parallel across two Macs, 57% faster decode
HankYeomans · x · 2026-11-04
A new mlx-serve PR by ivanfioravanti adds --dist tensor support for DeepSeek-V4.1 Flash across two Macs over Thunderbolt 5 RDMA (previously pipeline-only), using mlx-lm's SPMD layout with JACCL output summation. Measured on 2× vs 1× M3 Ultra with Lossless weights: DSpark decode 88 vs 56 tok/s (+57%, 116 on predictable text), plain decode 54 vs 41, prefill 1,608 vs 1,018 tok/s, TTFT 415ms → 268ms (-35%), 4 concurrent streams 94 vs 62 tok/s. LocalArena bench shows 85.6 tok/s at 8k context — faster than DwarfStar. PR merged.
More from coding & agent
- xAI engineer yacineMTB: agents work autonomously while I sleep, I wake up to breakthroughs — yacineMTB · 2026-11-04
- 8 Coding Agents on One RTX PRO 6000: Measured Throughput for Local Qwen Deployment — AIFlow_ML · 2026-11-04
- A start-to-finish workflow for AI dialogue scenes, from script to 1080p — DavidmComfort · 2026-11-04
- Agent37 launches $4.76/month AI agent sandboxes, undercutting Vercel's $249 by 98% — rohanpaul_ai · 2026-11-04
- Open-source Mercury Desktop ships a 5MB AI agent app for all platforms — HankYeomans · 2026-11-04
- Dev's Token Usage Skyrockets 417x in Six Months, Yet Fewer Lines of Code — ChemicalExcellent463 · 2026-11-04