176 KB C Program Runs 2.78T-Param Kimi K3 on a Single CPU with 8.24 GB RAM
techNmak · x · 2026-09-11
Someone wrote a 176 KB C program that runs the 2.78 trillion-parameter Kimi K3 on a single CPU with 8.24 GB of RAM — and the author of this thread dug into the repo to verify how. The checkpoint isn't shrunk: it's still 1.56 TB.
- MoE sparsity is the trick: 92 routed layers with 896 experts each, only 16 active per token — 82,432 routed experts occupying 1.447 TB (93% of the checkpoint), parked on NVMe instead of RAM
- Each expert is 17.56 MB; a token can require 1,472 expert fetches across layers, with the engine reading 25.83 GB of expert weights per token without resident hits
- The always-used 113 GB of weights are repacked to fit the tight memory budget
A striking example of combining MoE sparsity with NVMe streaming for "poor man's" large-model inference.
Related event: 2.78T-parameter Kimi K3 runs on a single CPU with 8.24GB RAM(4 posts)→
More from Infra
- Anthropic S-1 reveals $800B+ in off-books future compute commitments — GaryMarcus · 2026-10-27
- ~42% of Anthropic's ~11GW contracted compute is NVIDIA GPUs — ns123abc · 2026-10-27
- Anthropic's ~11 GW contracted compute estimated at ~42% NVIDIA GPUs, ~4.5 GW Google TPU — ns123abc · 2026-10-27
- EdgeRock-4B holds KV cache at 72 MiB across 256K context, 99.1% smaller than Qwen3.5-4B — alifcoder · 2026-10-27
- RTX 5080 price jumps from $1658 to $2318 in ten days on Amazon — Frizzy-MacDrizzle · 2026-10-27
- ComfyUI on 12GB VRAM: measured peaks show offloading turns 24s into 11 minutes — Impossible-Mail-4619 · 2026-10-27