4x 256GB M5 Ultra Mac cluster with exo serves 32 users at 2000 tok/s on Qwen 27B
StefanoGogioso · x · 2026-11-19
At Supercomputing 2026 (with SHI / Apple), exo labs demonstrated a cluster of 4x 256GB M5 Ultra Macs running exo distributed inference: 2000 tok/s aggregate throughput on Qwen 3.8 27B while serving 32 concurrent users at 62 tok/s each. Photos show stacked Mac Studio units at the show floor booth.
More from Infra
- Architect Labs unveils Frostbite, an agent-co-designed CPU claiming 4.6x perf per watt over Arm — IgorCarron · 2026-11-19
- Nvidia's unusual delay in filing its 10-Q sparks Wall Street intrigue — analyst explains why — TechCrunch AI · 2026-11-19
- Oxmiq Offers AI Chip IP and Tools to Indian Teams, Discusses Custom Silicon Economics — RajaXg · 2026-11-19
- How Airbnb runs coding agents on Bedrock: cost per task drives model picks and effort ceilings — AWS ML Blog · 2026-11-19
- SREGym 2.0 recreates the Microsoft Outlook outage: a latent gateway bug plus cascading failures frontier AI can't yet solve — ajratner · 2026-11-19
- Google ships a free fully-local AI notetaker for Mac powered by EmbeddingGemma 2 — dr_cintas · 2026-11-19