Running Qwen3.6-35B on a 16GB Mac
Senior_Nothing_4998 · reddit · 2026-07-18
This article details how the author extended antirez's ds4 runtime with a Metal backend to run Qwen3.6-35B-A3B on a 16GB M1 Pro.
The core challenge is that the Q4KS model size (20.8GB) exceeds the machine's 16GB capacity. To solve this, DS4 maintains a bounded routed experts cache within the unified memory and streams missing expert weights directly from the SSD.
Benchmark results on the M1 Pro 16GB show that the production build's four-run warm median achieved prefill 15.04 tokens/s and generation 9.77 tokens/s. Memory pressure remained normal, with no increase in swapout counts. The repository includes the runtime, ExpertMajor GGUF layout and converter, benchmark details, and links to the released GGUF files. While still experimental, the author confirms it is fully usable on a real 16GB machine.
More from Infra
- vLLM eyes Rust migration as inference engines race to rewrite the stack — AccBalanced · 2026-10-27
- Anthropic's S-1 reveals up to $567B compute spend and self-built data centers — jeremyakahn · 2026-10-27
- Resin lets an agent build standalone LLM inference engines; vLLM and SGLang optimize DeepSeek kernels differently — AccBalanced · 2026-10-27
- Local benchmark: Flint matches Qwen3.6 quality at 50/52 while running 3.6x faster on M4 Pro — RichardSocher · 2026-10-27
- Anthropic S-1: CSP marketplace revenue jumped from 11% to 48% in three years — AccBalanced · 2026-10-27
- Alchemy adds effect/Workflow support across Durable Objects, Workflows and Lambda Durable Functions — samgoodwin89 · 2026-10-27