Running Qwen3.6 on a 16GB Mac
andreaborio · hn · 2026-07-18
Demonstrates how to run Qwen3.6-35B-A3B on a 16GB M1 Pro. The core idea is using SSD-streamed MoE to stream model weights and experts on-demand from the SSD, rather than loading them entirely into memory at once.
Key takeaways from this approach:
- Makes otherwise oversized MoE models usable on low-memory machines.
- The focus is on the engineering trade-off between local inference and storage bandwidth, rather than a "model release."
- Offers direct practical value for users looking to run large models on laptops.
More from Infra
- Tabul AI launches Metal TreeSHAP to speed up Shapley values on Apple silicon — Scobleizer · 2026-07-22
- DeepSeek-V4-Flash tops out at 770 tok/s on one B300 in a vLLM batch test — Moreh · 2026-07-22
- NVIDIA starts shipping 102.4 Tbps Spectrum-6 switches for Vera Rubin AI factories — nvidia · 2026-07-22
- Apple publishes SOC 3 audit reports for Private Cloud Compute — throwfaraway4 · 2026-07-22
- Reddit GPU renters say existing platforms only give you two of three: code, recovery, fair billing — legendpizzasenpai · 2026-07-22
- The Sandboxing Manifesto: Secure Execution Environments for Agents — spirosoik · 2026-07-22