Kimi-K3 runs locally at 0.23 tok/s on a dual RTX 6000 PRO workstation
Aroochacha · reddit · 2026-07-29
A user reports getting Kimi-K3 running locally through a llama.cpp PR and shares very slow throughput numbers.
- Prompt evaluation: 40 tokens in 97.5s (0.41 tok/s)
- Main eval: 400 tokens in 1769.9s (0.23 tok/s)
- Total: 440 tokens in 1867s (31 minutes)
- The setup used a high-end workstation with dual RTX 6000 PRO 96GB cards and large NVMe RAID storage.
- The post also notes the next step is wiring the workstation into a 100GbE fabric for RPC use.
More from coding & agent
- Connected steps are not a workflow: nobody designs the handoff — usehive · 2026-10-16
- Making Blender animations with Opus, Astra and Argon: how you prompt still matters — chongdashu · 2026-10-16
- Running PolyStrata + Cline coding agent on a $0-fancy laptop at ~90 tok/s — rpwoerk · 2026-10-16
- How do you attribute agent gains: model training or infrastructure? — jrxna · 2026-10-16
- Feed Claude your books, blogs and top engineers' code for claude.md — 'it now writes better code than me' — blaizedsouza · 2026-10-16
- Open-source /label plugin auto-names and colors Claude Code sessions in one step — hermanschutte · 2026-10-16