Clover-35B-A3B Quantized to 29.4 GB Runs at ~42 tok/s on a 64 GB Mac
viveky259 · reddit · 2026-10-31
A detailed first-party writeup of running Clover-35B-A3B-Preview locally on a 64 GB Mac.
Fitting it
- Mixed-precision quantization: 6-bit weights with selected tensors in BF16, 29.4 GB total; no training/pruning/distillation
Speed (MLX on Apple GPU)
- Compiled computation reuse, block-based generation with blocks of 8, fused preview + token selection into one compiled op
Results
- Six runs: 40.30–44.30 tok/s, averaging 42.46 (vs 40.05 pre-fusion)
- Load time 124 s; MLX peak memory 29.8 GB; weights on external SSD, macOS swap still active
Unresolved: quality
- Custom 40-case coding/summarization eval: fast config 21/40, repaired candidate 32/40, conservative 35/40
- Failures included malformed Python, wrong return types, a summary that reversed a restriction—latest speedups don't prove these are fixed
Bottom line: promising local throughput, unfinished quality work.
More from Infra
- Cloudflare Durable Objects now defer code updates up to 300s by default — irvinebroque · 2026-10-31
- Intel plunges 29%, falling below its 2000 dot-com era peak after 26 years — mitchdeg · 2026-10-31
- Liquid Agent brings fully local LFM3 agent to Linux with one-command install — JosephJacks_ · 2026-10-31
- A fully local office AI stack: MiniCPM, Qwen3, LFM2.5 and Whisper — HowDevelop · 2026-10-31
- a16z: GPU rental prices up 106% in 3 months while token prices fall 58% — a16z · 2026-10-31
- vLLM nightly adds PLE disk offload, with a telling [Qwen4Exp] tag on the PR — rmhubbert · 2026-10-31