Apple's MLX runs an 816B model across four Mac Studios at 36 tokens/sec
JeremyCMorgan · x · 2026-10-23
Apple's MLX team reports over 36 tokens per second from an 816B-parameter model distributed across four Mac Studios, with the method and power measurements published. The author raises the follow-up question: what does this setup cost per token? A rare official data point for distributed Mac-based local inference of frontier-scale models.
More from Infra
- HFT giants now fight for compute: Jane Street-backed Thinking Machines hits $40B valuation — dinabass · 2026-10-23
- Bloomberg: Tencent, Legend Capital and CPE circle Huawei Ascend chips, same names near DeepSeek — YvesMulkers · 2026-10-23
- 200 concurrent LLM agents on a 1GB, $6/month VM — the bottleneck was our own save locks — swarmagent · 2026-10-23
- Blaise's new book hailed as year's best; his space-GPU Project Suncatcher just launched — paulnovosad · 2026-10-23
- Free browser-based Jetson farm runs Cosmos3-Edge on 8GB via INT4 at 152 tok/s — No-Dragonfly6246 · 2026-10-23
- Modded 32GB RTX 4080 unlocked: BF16 jumps from 92 to 184 TFLOPS — Desperate-Sir-5088 · 2026-10-23