Running a 2.78T-parameter Kimi K3 inference on a single CPU in 8.24 GB of RAM
gekobraa · x · 2026-09-10
The OSPs project claims to run a 2.78-trillion-parameter Kimi K3 inference on a single CPU using just 8.24 GB of RAM — no GPU, no BLAS, no framework. If verified, this showcases extreme memory-mapped/quantized inference, though the post offers no throughput or latency details.
More from Infra
- Screenshot surfaces rare admission of 72-hour KV cache limits in V4-era architecture — zephyr_z9 · 2026-09-10
- DeepSeek cut per-token KV cache size by 54x in nine months — zephyr_z9 · 2026-09-10
- vLLM Ships Full Support for DeepSeek-V4.1-Flash's New Architecture — vllm_project · 2026-09-10
- No one matches its inference economics; commentator suggests 10-20% cut from infra providers — zephyr_z9 · 2026-09-10
- Apple A20 Pro Neural Engine projected to hit 140+ TFLOPS, a third of an A100 for local inference — AIFlow_ML · 2026-09-10
- V4.1 Flash KV cache is 437x smaller than V1, easing China's memory bottleneck — ChrisGPT · 2026-09-10