Bypassing RAM Limits: Open-Source C Engine Runs 2.78T Kimi K3 Locally
ab2377 · reddit · 2026-08-03
A new GitHub project named WASTE introduces an embeddable C inference engine with zero third-party dependencies. It allows users to run the massive 2.78-trillion-parameter Kimi K3 model even when system RAM is insufficient.
The core mechanism keeps the model trunk in memory while streaming activated experts directly from NVMe drives, utilizing the remaining RAM as a bounded expert cache. This provides a promising solution for local deployment of ultra-large MoE models.
Related event: Open-source engines run Kimi K3 on low-end devices(7 posts)→
More from Infra
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16
- NVIDIA quietly delists the entry-level RTX 5050 from its marketplace, following the RTX 5090 — chemist_slime · 2026-10-16
- Grok claims 816B MoE model runs locally on dual M5 Ultra Mac Studios via MLX — grok · 2026-10-16
- Same 1,000 Cloudflare Durable Objects: $4,050/month cut to $22 with one trick — threepointone · 2026-10-16
- NVIDIA ships FP4-quantized Qwen3.8-Flash for local use, trending on Hugging Face — nvidia · 2026-10-16
- Baidu's Virtual Head Attention tames the KV cache, FLOPs, quality triangle in long-context LLMs — baidu · 2026-10-16