FULL STORY
Kimi K3 Open Source and Extreme Edge Inference
Moonshot open-sourced the massive Kimi K3 model, leading the community to develop innovative inference engines that enable it to run on low-end consumer hardware, even using pure C with just 8GB of RAM.
2026-08-01 ~ 2026-08-05 · 3 episodes · 14 posts
Episode 1 · Moonshot's Open-Source Kimi K3 Lands on Multiple Platforms (2026-08-01, 4 posts)
Moonshot AI's open-source model Kimi K3, featuring 2.8 trillion parameters and a 1 million token context window, is now available on OpenRouter and Microsoft Azure AI Foundry. Together AI has also released a developer guide to facilitate its deployment, expanding the model's ecosystem.
- Moonshot's 2.8T Parameter Kimi K3 Lands on Microsoft Azure AI Foundry — ccerrato147 · 2026-08-01
- Kimi K3 Hits OpenRouter: 2.8T Parameters, 1M Context Length — AccBalanced · 2026-08-01
- Kimi K3 Lands on Microsoft Azure: Are Open Models Eroding the Model Moat? — ccerrato147 · 2026-08-02
- Together AI Releases Kimi K3 Developer Guide: 2.8T Params Rivaling GPT 5.6 — togethercompute · 2026-08-02
Episode 2 · Open-source engines run Kimi K3 on low-end devices (2026-08-01, 7 posts)
Recent open-source engines like WASTE enable running the 2.78T-parameter Kimi K3 on consumer devices via innovative weight-loading mechanisms, breaking the traditional memory barrier.
Confirmed
- WASTE engine: Developer Marco Bambini released WASTE (Weight-Aware Streaming Tensor Engine), a dependency-free C inference engine that runs the unmodified Kimi K3 in 29GB RAM, achieving 0.5 TPS on a consumer laptop.
- Core mechanism: Streams weights directly from NVMe when RAM is insufficient. @机器之心 tested on a 128GB unified-memory Mac, successfully loading 1.56TB of official MXFP4 weights via streaming, with very slow generation but functional responses.
- C99 minimal inference: A developer wrote a minimal C99 engine for Kimi K3, leveraging 93% on-demand parameters, running on a single CPU with 8.24GB RAM.
Why it matters
- Lower deployment barrier: These projects show that software optimizations can bypass expensive hardware limits, allowing individuals to run massive models on ordinary laptops or old devices, promoting local AI adoption.
- Full 2.78T-parameter Kimi K3 Runs on Consumer Laptop via NVMe Streaming — rickasaurus · 2026-08-01
- Running 1.6TB Kimi K3 Weights: 128GB Mac vs 80x RTX 5090 Cluster — 机器之心 · 2026-08-01
- Open-Source Engine 'Waste' Runs Kimi K3 on Just 29GB of RAM — galapag0 · 2026-08-01
- Run 2.78T-Parameter Kimi K3 on a Laptop: WASTE Engine Achieves 0.5 TPS — zainhas · 2026-08-01
- Running Kimi K3 in Pure C99: Low-Config Inference on 8GB RAM — FareedKhan557 · 2026-08-02
- Bypassing RAM Limits: Open-Source C Engine Runs 2.78T Kimi K3 Locally — ab2377 · 2026-08-03
- Running 2.78T Parameter Kimi K3 on a Single CPU with 8GB RAM — Saboo_Shubham_ · 2026-08-03
Episode 3 · Kimi K3 Runs on Single CPU with 8GB RAM via Pure C (2026-08-03, 3 posts)
A popular GitHub project demonstrates how to run the 2.78 trillion parameter Kimi K3 model using pure C on a single CPU and 8GB of RAM by leveraging SSD streaming inference, significantly lowering hardware barriers.
- Run 2.78T Parameter Kimi K3 on a Single CPU in 8.24GB RAM — Saboo_Shubham_ · 2026-08-03
- Kimi K3 reportedly runs on an 8GB CPU setup by streaming experts from SSD — porAssass · 2026-08-04
- Run Kimi K3 in Pure C: 2.78T Parameters on a Single CPU with 8GB RAM — gethackteam · 2026-08-05