2x256GB M5 Ultra tested: two separate models beat clustering, GLM q4 beats DeepSeek API for coding
Viktri1 · reddit · 2026-11-08
The author shares hands-on results running local LLMs on two 256GB Mac Studio M5 Ultra machines.
- Clustering loses to two separate models: tensorfold/exos-style clustering suffers from poor concurrency and inconsistent decode gains.
- Current setup: Qwen Flash q6 as the fast front end + GLM 3.5 Flash q4 for deep reasoning; the author finds GLM at q4 beats even the official DeepSeek API for coding.
- Memory bottleneck: 256GB can't fit GLM with 500k+ context — stuck at 260k with max 3 agents; a 32GB hot cache caused RAM exhaustion and a 70% speed drop. Conclusion: the 512GB variant is worth it, and future hardware should make KV cache cheaper on RAM.
More from Infra
- Musk mocks $150k Mac Studio local LLM setup that's worse than a $200/mo subscription — elonmusk · 2026-11-08
- Bank dev tests 1.7B local model offline: all traffic stayed on localhost, emails in 2.5s — Ok_Resolve4741 · 2026-11-08
- Musk endorses orbital AI compute pitch: 100 GW/year could add $1T annual revenue to SpaceX — elonmusk · 2026-11-08
- BF16 GPU TFLOPs comparison table proves its value over time, says HF dev Zach Mueller — TheZachMueller · 2026-11-08
- Starlink adds 1.6M consumer subscribers in a quarter, hits 13.6M worldwide — XFreeze · 2026-11-08
- Leeds AI datacentre row: water use equals just 5 car washes, 1,000+ jobs at stake — danbri · 2026-11-08