Palm-Infra open-sourced: Run 284B models on MacBook via SSD offloading
dr_cintas · x · 2026-08-26
Tencent's Palm-Infra framework enables running massive MoE models on Apple Silicon by streaming experts directly from SSD. Benchmarks show DeepSeek-V4-Flash (284B) decoding at 5.71 tps and a 122B model at 16.53 tps on an M5 Pro with 20GB memory, achieving capabilities previously impossible with llama.cpp.
Related event: Tencent Open-Sources Palm-Infra, Running 284B Models on a MacBook(2 posts)→
More from Infra
- ASML CFO says US Match Act restricting DUV sales to China would backfire — pstAsiatech · 2026-10-15
- Bloomberg breakdown: a 1GW Texas AI data center may need nearly $3B for overruns alone — rohanpaul_ai · 2026-10-15
- 7 System Design Algorithms to Know: Merkle Trees to CAP/PACELC — goyalshaliniuk · 2026-10-15
- MiniCPM-V 4.7 runs a 35B MoE multimodal model on phones with just 3GB memory — nevrekaraishwa2 · 2026-10-15
- Sipeed MaixStation product page goes live for edge AI dev — xlltt · 2026-10-15
- Nvidia Is Now Worth More Than the Entire Indian Stock Market — theteknosaur · 2026-10-15