Run 2.78T-Parameter Kimi K3 on a Laptop: WASTE Engine Achieves 0.5 TPS
zainhas · x · 2026-08-01
The open-source project WASTE (Weight-Aware Streaming Tensor Engine) enables running the massive 2.78-trillion-parameter Kimi K3 model on a consumer laptop.
- Core Mechanism: Streams activated weights directly from NVMe, bypassing physical RAM limitations.
- Performance: Achieves inference speeds of about 0.5 tokens/s on a 64GB RAM machine.
- Features: Built as a dependency-free, embeddable C inference engine, now available on GitHub.
Related event: Open-source engines run Kimi K3 on low-end devices(7 posts)→
More from Infra
- China's AI boom drives up to 660% profit surges for domestic chip firms; GPU maker MetaX turns profitable — pstAsiatech · 2026-10-16
- Qdrant tests Perplexity's visual PDF embeddings: 21x smaller vectors, 99.9% after rescoring — jrxna · 2026-10-16
- When Does an Agent Actually Need an API Gateway? Reddit Weighs Direct Calls vs. Unified Endpoints — TurbulentStiffness · 2026-10-16
- Taiwan to build $4B AI manufacturing complex in Texas to back Nvidia and TSMC — Polymarket · 2026-10-16
- When the KV Cache Is Slower Than the GPU: Remote KV Tiers Backfire in Production — MultiverseComputingCAI · 2026-10-16
- GitHub rewrote the 800k-line Copilot runtime in Rust using AI agents, cutting CPU and memory by over 90% — bibryam · 2026-10-16