GLM-5.3-Flash 320B served from two Mac Studios: 137 tok/s single-user, 32 users, zero errors
AIFlow_ML · x · 2026-11-23
Developer @sashmit ran the 320B GLM-5.3-Flash on two M5 Ultra 512GB Mac Studios connected by a single Thunderbolt 5 cable:
- 137 tok/s for a single user with TensorFold and drafts on; 105 tok/s with drafts off
- 32 concurrent users at 260 tok/s total throughput, with zero errors in a 30-minute soak test
- A single M5 Ultra 512GB delivers 89.5 tok/s
Takeaway: a desk setup can now serve a frontier-size MoE to 32 people — no server room required. Figures are source-measured, not re-run. Source: AIFlowML on X.
More from Infra
- RSIBench-Infra from ByteDance Seed and Tsinghua tests if agents can optimize production-scale AI infrastructure — geoffwolfe · 2026-11-23
- Building a local-first creative lab: M5 Ultra Mac Studio with 256GB RAM — davidhoang · 2026-11-23
- Oracle allegedly hid 50% server price surge, scaling back OpenAI compute project — SumitGup · 2026-11-23
- Big Short's Steve Eisman Warns Nvidia's Circular Financing Could Unravel After Huang's Exit — SumitGup · 2026-11-23
- Cloudflare Workers Traces goes paid Dec 1 — lower head_sampling_rate to cut the bill — yihui_indie · 2026-11-23
- $19,068 Threadripper 9980X workstation order screenshot backs 'people, compute, AI tokens' thesis — jacob_posel · 2026-11-23