Running GLM-5.2 and Kimi K3 Locally on RTX 5090: Hits 129 tok/s
markjeffrey · x · 2026-07-31
A user shared local inference benchmarks using a half-price RTX 5090 consumer setup.
- GLM-5.2: Achieved a decode speed of 129 tok/s.
- Kimi K3: Reached 108 tok/s.
- Context: This test responds to a previous frontier deployment by the tinygrad team achieving 120 tok/s for GLM-5.2 on AMD boxes, proving that consumer-grade GPUs offer highly cost-effective performance for running large models.
More from Infra
- Trading 2 RTX 5090s for 4 R9700s: does doubled VRAM pay off? — poofph · 2026-10-16
- Running PolyStrata + Cline coding agent on a $0-fancy laptop at ~90 tok/s — rpwoerk · 2026-10-16
- TensorFold 1.0.12 runs 120B vision model on 1-2 DGX Sparks, adds Mac image input — HankYeomans · 2026-10-16
- vLLM brings frontier model local deployment to NVIDIA RTX Spark PCs — vllm_project · 2026-10-16
- NVIDIA signals $2B in new European AI deals at GTC Berlin, totaling ~$10B in a year — CurieuxExplorer · 2026-10-16
- Making Your Data Ready for Agentic AI: trust first, then semantics, then writes — bibryam · 2026-10-16