llama-modchip Runs DeepSeek's 284B Model Entirely in 64GB RAM With Zero GPUs
solyarisoftware · x · 2026-11-09
The open-source llama-modchip project runs DeepSeek's 284B model entirely in RAM with no GPU: the full model fits in 64GB of memory at 9-15 words per second with 256K context, while 16-32GB laptops run 30B MoE models at 25-111 tokens per second. It works on Intel, AMD, and Apple chips, using techniques like SwiGLU-H2 and Stockfish-style speculative decoding. A model of this size previously required a rack of GPUs; the Apache-2.0 repo has drawn 584 stars.
More from Infra
- Satellite imagery shows China's 'Stargate' Phase 1 delivered, Phase 2 due Q2 2027 — teortaxesTex · 2026-11-09
- Getting Omarchy running on Snapdragon Surface Pro: custom kernel plus SSH tunnel for OpenClaw — vincent_koc · 2026-11-09
- Ex-PlayStation Boss: A $300 PS5, Not a $1,200 PS6, Would Be True Innovation — aakashgupta · 2026-11-09
- Rumor: Megaspeed's SEA datacenter shell route triggered US crackdown, B300 spot prices soar — basedjensen · 2026-11-09
- NVIDIA's AutoCompress Beats Minitron by Up to 6.57 Points With Hardware-Guided LLM Pruning — nvidia · 2026-11-09
- China sets fiber record: 64 Tbps each way over 1,007 km of hollow-core fiber — AIFlow_ML · 2026-11-09