What to run on a 96GB M3 Ultra? Qwen3.5-122B-A10B hits ~1000 tok/s locally
infieldmitt · reddit · 2026-09-17
A Reddit user asks what to run locally on a 96GB M3 Ultra Mac Studio, wanting both strong coding and emotionally intelligent writing. They complain current options are either 30B or way too big; running two Qwen3 27B instances to use the RAM feels futile since a faster MoE can preprocess quickly anyway. They just downloaded Qwen3.5-122B-A10B—up to 1000 tok/s and noticeably better prompt comprehension than smaller models—but worry the "3.5" naming means something inferior.
More from Infra
- To dethrone Postgres, new databases must prioritize ecosystem integrations from day one — mgill25 · 2026-11-02
- ESP32-powered 4-inch touchscreen builds a TUI dashboard for monitoring a DGX inference cluster — CATLLM · 2026-11-02
- OpenRouter crosses 10 quadrillion annualized token run rate, doubling every 10.5 weeks — deedydas · 2026-11-02
- Palmer Luckey asks: can you use datacenter RAM for things prohibited by the operator's terms of service? — avaitopiper · 2026-11-02
- Disable MTP for 20-30% faster local LLM inference on limited RAM — InternalMode8159 · 2026-11-02
- Kioxia up 216% in a year but down 68% from its June 2026 peak — zephyr_z9 · 2026-11-02