Full 2.78T-parameter Kimi K3 Runs on Consumer Laptop via NVMe Streaming
rickasaurus · x · 2026-08-01
Developer Marco Bambini introduced WASTE (Weight-Aware Streaming Tensor Engine), a clean C-based inference engine that successfully runs the unmodified, 2.78-trillion-parameter Kimi K3 model on a consumer laptop.
The core of this approach is weight-aware streaming: it bypasses VRAM limits by streaming only the activated experts directly from NVMe storage. The process requires no distillation, pruning, or cloud connection, preserving the full open weights. While currently slow, it proves the technical feasibility of running massive MoE models on local hardware.
Related event: Open-source engines run Kimi K3 on low-end devices(7 posts)→
More from Infra
- China's AI boom drives up to 660% profit surges for domestic chip firms; GPU maker MetaX turns profitable — pstAsiatech · 2026-10-16
- Qdrant tests Perplexity's visual PDF embeddings: 21x smaller vectors, 99.9% after rescoring — jrxna · 2026-10-16
- When Does an Agent Actually Need an API Gateway? Reddit Weighs Direct Calls vs. Unified Endpoints — TurbulentStiffness · 2026-10-16
- Taiwan to build $4B AI manufacturing complex in Texas to back Nvidia and TSMC — Polymarket · 2026-10-16
- When the KV Cache Is Slower Than the GPU: Remote KV Tiers Backfire in Production — MultiverseComputingCAI · 2026-10-16
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16