Kimi K3 reportedly runs on an 8GB CPU setup by streaming experts from SSD
porAssass · reddit · 2026-08-04
A Reddit post breaks down a project that claims to run Moonshot’s Kimi K3 on a regular CPU with as little as 8GB of RAM.
- The key idea is that the full 1.5TB checkpoint is never loaded into memory at once.
- The engine uses MoE sparsity: only 16 of 896 experts activate per token.
- Expert weights are streamed from NVMe SSD, with an LRU cache for recently used experts.
- Computation happens directly on compressed MXFP4 weights.
- The whole inference stack is written in portable C99, without PyTorch, CUDA, TensorRT, or BLAS.
The author notes the system is extremely slow — seconds per token on low-memory hardware — but argues the design is interesting as a systems-engineering proof of concept for future inference engines.
Related event: Kimi K3 Runs on Single CPU with 8GB RAM via Pure C(3 posts)→
More from Infra
- Trading 2 RTX 5090s for 4 R9700s: does doubled VRAM pay off? — poofph · 2026-10-16
- Running PolyStrata + Cline coding agent on a $0-fancy laptop at ~90 tok/s — rpwoerk · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16
- vLLM brings frontier model local deployment to NVIDIA RTX Spark PCs — vllm_project · 2026-10-16
- NVIDIA signals $2B in new European AI deals at GTC Berlin, totaling ~$10B in a year — CurieuxExplorer · 2026-10-16
- Making Your Data Ready for Agentic AI: trust first, then semantics, then writes — bibryam · 2026-10-16