Kimi K3 reportedly runs on 80 RTX 5090s with 2.56 TB of VRAM
sandyyevans · reddit · 2026-07-28
The Ning team reportedly deployed Kimi K3 on 80 RTX 5090s for local inference, using 10 nodes × 8 cards with 2.56 TB of total VRAM. The setup runs with FP4 weights for now, avoids HBM, NVLink, and InfiniBand, and relies on the 5090’s GDDR7 plus 25GbE networking.
Reported throughput is about 20 tok/s for single-stream inference, with the team saying the deployment is not yet optimized. The poster argues the choice may be partly a publicity stunt, but sees it as a sign that running frontier-scale models locally could become more realistic as consumer and datacenter GPUs keep improving.
Related event: Kimi K3 Runs on 80 RTX 5090 GPUs Without HBM(5 posts)→
More from Infra
- x402 is emerging as a pay-per-use GTM rail for agentic AI tools — kleffew94 · 2026-07-28
- Moonshot reportedly trained Kimi K3 on Nvidia Blackwell chips for its next model — RebeccaBellan · 2026-07-28
- KLA’s call will matter more for AI demand than for the revenue beat — tengyanAI · 2026-07-28
- Morgan Stanley estimates GPU rental businesses can reach 70% incremental margins — firstadopter · 2026-07-28
- Compute demand stays strong while capital turns cautious, says Ben Bajarin — BenBajarin · 2026-07-28
- TensorLib is a pure-C deep learning library that trains tiny language models on CPU — Intelligent_Nose_791 · 2026-07-28