Reddit user gets about 4 tokens/s running Kimi K3 on a 2×5090 home lab
iVoider · reddit · 2026-07-30
A Reddit post reports first home-lab results for Kimi K3: about 4 tokens per second on a machine with 768GB DDR5 and 2×5090 GPUs. The author also says prefill speed on long prompts reaches 50–70 tokens per second.
One oddity in the test is that decoding throughput appears to increase over time, which the author suspects may be due to warmup or swapping behavior. The benchmark tool crashed, so no formal comparison is shared.
More from Infra
- NVIDIA launches RTX Spark for local AI on Windows PCs — ostrisai · 2026-10-16
- SkyRL ships Day-0 RL support for Beam with +62% LoRA decode throughput in vLLM — vllm_project · 2026-10-16
- DFlash2 delivers up to 3.03x decode-only speedup for Beam across five benchmarks — vllm_project · 2026-10-16
- vLLM's Beam serving kernels cut GB200 forward time from 8.15 to 6.47 ms at TP8 batch 128 — vllm_project · 2026-10-16
- Reflection's 501B MoE Beam hits 463 tok/s on 8x GB200 with NVFP4 at day 0 — zhyncs42 · 2026-10-16
- Gemma 4 31B QAT on one 24GB card: 46 tok/s plain, ~80 with draft model on RTX 4090 — Subject_Poet_3861 · 2026-10-16