Run 2.78T-Parameter Kimi K3 on a Laptop: WASTE Engine Achieves 0.5 TPS
zainhas · x · 2026-08-01
The open-source project WASTE (Weight-Aware Streaming Tensor Engine) enables running the massive 2.78-trillion-parameter Kimi K3 model on a consumer laptop.
- Core Mechanism: Streams activated weights directly from NVMe, bypassing physical RAM limitations.
- Performance: Achieves inference speeds of about 0.5 tokens/s on a 64GB RAM machine.
- Features: Built as a dependency-free, embeddable C inference engine, now available on GitHub.
Related event: Open-Source WASTE Engine Runs Trillion-Parameter Kimi K3 on Laptops(3 posts)→
More from Infra
- Data Center Lobby's AI-Generated Ad Campaign Operates with Black Box Funding — ShakeelHashim · 2026-08-01
- 2026 Predicted as Best Year for DePIN: AI Drives Decentralized Compute and Agent Economy — 0xJeff · 2026-08-01
- Cut Your $200/Month AI Bill: 10 Steps to Local AI Deployment — jason_mayes · 2026-08-01
- Local MXFP4 Testing Reveals Inconsistent Quality Among OpenRouter DS4 Flash Providers — antirez · 2026-08-01
- DeepSeek V4 Flash local benchmark nearly matches top frontier models from 5 months ago — joorklee · 2026-08-01
- New Method Pre-routes MoE Layers to Optimize I/O for Edge Streaming — dai_app · 2026-08-01