Running 284B DeepSeek on 64GB MacBook: Lossless quality confirmed

cowboy-bebob · reddit · 2026-08-22

Notes on running DeepSeek-V4-Flash (284B) on a 64GB MacBook:

• Memory: Model is 165GB. Runs via partial weight loading and SSD offloading.
• Quality: No quality drop. Perplexity score 6.1250 vs official 6.1262.
• Speed: 11 tps for 2-bit, 1-2 tps for higher quality.
• Cache: 8GB cache was faster than 32GB. macOS native file caching outperforms app-level allocation.
• Optimizations: Speculative decoding and prefetching slowed it down due to disk bottleneck.

Code, logs, and failed experiments are available in the repo.

Original post →

More from Infra

• Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
• Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16
• vLLM ships Day-0 LoRA RL for Muse Spark Open, lifting AIME from 0.56 to 0.85 — vllm_project · 2026-10-16
• DSpark draft model for Muse Spark Open proposes 7 tokens per pass with 23-42% acceptance — vllm_project · 2026-10-16
• Cloudflare Stream's WebRTC ultra-low-latency live streaming is now Generally Available — ritakozlov · 2026-10-16
• YC-backed Overshoot launches inference optimization, claims $5M saved for AI teams — ycombinator · 2026-10-16

Infra channel →