Is 5t/s KIMI K3 on a Single 5090 Feasible?
MLDataScientist · reddit · 2026-08-20
A user questions the feasibility of running KIMI K3 at 5t/s on a single RTX 5090 with 4TB NVMe. Calculations suggest that while theoretically possible with high expert cache hit rates, real-world random read speeds on SSDs make it unlikely. The post lists several relevant backend projects like llama-kimibri and ds4 for further testing.
More from Infra
- LLM training speedup 19% by switching to built-in GELU — rasbt · 2026-08-20
- Data centers should be civic infrastructure built with pride and beauty — aronchick · 2026-08-20
- Databricks AI Extract achieves 95% accuracy on PDF field extraction — jefrankle · 2026-08-20
- Animation claims AI data center water use panic is over almost nothing — aronchick · 2026-08-20
- Virtuals: Robotics data can't be scraped like the web — ai · 2026-08-20
- New Disaggregation for Hybrid Linear Models on Cerebras CS-4 — AccBalanced · 2026-08-20