RTX PRO 6000 Hits 2900 tok/s Running 30B Models
HankYeomans · x · 2026-08-13
A user shared benchmark results for the NVIDIA RTX PRO 6000 GPU. Running the Nemotron-3.5-Lightning-30B-A3B-NVFP4 model, the card achieved an aggregated throughput of 2900 tok/s and secured 8.1M of KV cache. The author noted that while the card is expensive, it is worth the price.
Related event: RTX PRO 6000 Hits 2900 tok/s in 30B Model Local Inference Test(2 posts)→
More from Infra
- MLCommons launches MLPerf Endpoints, a benchmark that measures inference endpoints the way buyers actually purchase them — TheKanter · 2026-10-16
- Neocloud M&A decoded: CoreWeave, Nebius, Crusoe, Nscale all bet margins vanish beyond GPU scarcity — AccBalanced · 2026-10-16
- AI-Driven Timing-Aware Power Optimization Uses Learned Ranking With Baseline-Preserving Repair — Amit Sharma · 2026-10-16
- Nscale AI datacenter plagued by coolant leaks, transformer fire, 10 safety violations — saranormous · 2026-10-16
- WSL 3.0 ships with native containers and Linux 6.18 kernel, syscall overhead down ~16% — solyarisoftware · 2026-10-16
- Are 22 GB modded RTX 2080 Ti cards a viable budget local LLM option? — Dave86ch · 2026-10-16