Run 284B-parameter models locally on a MacBook
socialwithaayan · x · 2026-08-26
Developers demonstrated the ability to run large-scale models locally on a MacBook: DeepSeek-V4-Flash decodes at 5.71 tokens/sec on an M5 Pro with 20GB memory, and a 122B model achieves 16.53 tps. This is enabled by Palm-Infra, which streams experts directly from the SSD, allowing models that llama.cpp cannot even load to run. The project is 100% open source.
Related event: Tencent Open-Sources Palm-Infra, Running 284B Models on a MacBook(2 posts)→
More from Infra
- NVIDIA releases Nemotron 3.5 Bolt 74B MoE that fits in 24GB VRAM at 2-bit — NerdyRodent · 2026-10-16
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16