Kimi K3 reportedly runs on an 8GB CPU setup by streaming experts from SSD
porAssass · reddit · 2026-08-04
A Reddit post breaks down a project that claims to run Moonshot’s Kimi K3 on a regular CPU with as little as 8GB of RAM.
- The key idea is that the full 1.5TB checkpoint is never loaded into memory at once.
- The engine uses MoE sparsity: only 16 of 896 experts activate per token.
- Expert weights are streamed from NVMe SSD, with an LRU cache for recently used experts.
- Computation happens directly on compressed MXFP4 weights.
- The whole inference stack is written in portable C99, without PyTorch, CUDA, TensorRT, or BLAS.
The author notes the system is extremely slow — seconds per token on low-memory hardware — but argues the design is interesting as a systems-engineering proof of concept for future inference engines.
Related event: Open-Source Project Claims Kimi K3 Can Run on CPU With 8GB RAM(2 posts)→
More from Infra
- Exa says its web index has 80B pages and is on track for Google-scale in 2027 — garrytan · 2026-08-04
- OpenCode Go says it processed 6T tokens in a single day, led by DeepSeek models — ycombinator · 2026-08-04
- Cloud and AI vendors are creating expensive lock-in and uncontrolled future risks — DavidLinthicum · 2026-08-04
- Swarms Cloud adds saved workflows, session persistence, and 1,500+ models — KyeGomezB · 2026-08-04
- SK Hynix prepares to break ground on $3.87B Indiana HBM packaging fab — rwang07 · 2026-08-04
- Subnet 44 expands around Satori, a 7B vision-language model for grounding — richdotca · 2026-08-04