Running Qwen3.6 on a 16GB Mac
andreaborio · hn · 2026-07-18
Demonstrates how to run Qwen3.6-35B-A3B on a 16GB M1 Pro. The core idea is using SSD-streamed MoE to stream model weights and experts on-demand from the SSD, rather than loading them entirely into memory at once.
Key takeaways from this approach:
- Makes otherwise oversized MoE models usable on low-memory machines.
- The focus is on the engineering trade-off between local inference and storage bandwidth, rather than a "model release."
- Offers direct practical value for users looking to run large models on laptops.
More from Infra
- Cloud capacity crisis is now an AI architecture problem, argues new newsletter — DavidLinthicum · 2026-10-26
- SpaceX files $40B GPU financing commitment with Apollo, already at 1.7GW — XFreeze · 2026-10-26
- Meta plans to raise up to $95B to fund massive AI expansion — Polymarket · 2026-10-26
- Should the community archive open model weights in case Hugging Face gets restricted? — Tall-Ad-7742 · 2026-10-26
- What happens to neocloud GPUs when the AI datacenter debt bubble bursts? — bowdoin-yale · 2026-10-26
- Nebius CEO: Debt Markets Closed for Neoclouds Last Week, Leaning on Customer Advances — zephyr_z9 · 2026-10-26