Banach: open-source engine streams 1T-parameter Kimi K2 from NVMe on a 64GB desktop
gautamrbharadwaj · reddit · 2026-11-05
A developer released Banach, an open-source inference engine that runs models bigger than RAM by keeping active experts in memory and streaming the rest from NVMe. Kimi K2 has 1T parameters but only 32B active per token.
- Measured: a 22GB model on 16GB RAM — llama.cpp OOMs, Banach is 2.2x faster at prompt processing with byte-identical output
- Estimated: Kimi K2 (382GB weights) at 3.6 tok/s on a 64GB desktop with an RTX 5080; author notes it's a plan-based estimate that under-promised 2.5x on the measured model
- Install: pip install banach
More from Infra
- imec Ball-Lens Scheme Cuts Photonic Chip-to-Board Coupling Loss to 3.4dB via Substrate Vias — jwt0625 · 2026-11-05
- Google Paper Details 81-Satellite Cluster for Orbital AI Datacenters — sebkrier · 2026-11-05
- e/acc figurehead Beff Jezos announces going all-in on space-grade thermodynamic chips — beffjezos · 2026-11-05
- citrini: being part of AI supply chain no longer guarantees 100% yearly stock gains — citrini · 2026-11-05
- Building orbital data centers to dodge regulators is 'completely unserious' — zacharynado · 2026-11-05
- Verda Launches Inference Preview with Day-0 Access to Kimi K3.1 Pro — Kimi_Moonshot · 2026-11-05