Mac Studio M3 Ultra 512GB for local LLMs: 20-90 tok/s, and how to decide if it's worth it
rez0__ · x · 2026-10-28
Security researcher Jhaddix says his Mac Studio M3 Ultra 512GB was worth it, running DeepSeek Flash and GLM 5.3 at 20-40 tok/s with 1M context 24/7. rez0 agrees, getting 80-90 tok/s from a 4-bit quant with thinking on a 256GB Mac Studio via omlx, using 200k context via uniichat. Their shared method: price out the hosted model you need per year, compare against hardware cost, and mix local models, frontier subscriptions, and pay-as-you-go APIs.
More from Infra
- preowned turns macOS built-in Apple models into an OpenAI-compatible local API — pcuenq · 2026-10-28
- 2,100+ tok/s/user for DeepSeek V4.1 Flash on 8× AMD MI355X, 2.8x faster than open-source — JiaZhihao · 2026-10-28
- Half of Agent Infra users are no longer human: Cloudflare, Vercel and Supabase numbers — FinanceYF5 · 2026-10-28
- Phonon on-device TTS hits 80K downloads with PyTorch-free Rust runtime — mattturck · 2026-10-28
- Agent infra as the sure bet: Manus runs 1.5M databases on TiDB, investors say — FinanceYF5 · 2026-10-28
- Octen-WebGR Indexes 201M Web Pages in a 4B Model's Parameters, Beats Dense Retrieval and Frontier LLMs at Navigational Search — _reachsumit · 2026-10-28