RTX PRO 6000 Hits 2900 tok/s Running 30B Model Locally
HankYeomans · x · 2026-08-13
A developer shared benchmark results for local inference using the NVIDIA RTX PRO 6000 GPU.
- Model & Quantization: Running the Nemotron-3.5-Lightning-30B-A3B-NVFP4 model.
- Performance: Achieved an aggregated throughput of 2900 tok/s and secured 8.1M of KV cache.
- Verdict: The author notes that while the GPU is expensive, it is worth the price.
Related event: RTX PRO 6000 Hits 2900 tok/s in 30B Model Local Inference Test(2 posts)→
More from Infra
- New hardware day: local LLM deployment the moment it arrives — lxfater · 2026-10-16
- Together AI's score centering fixes RL training collapse under quantized inference — nrehiew_ · 2026-10-16
- Inference is splitting prefill and decode across specialized chips, not just H100s — AccBalanced · 2026-10-16
- Rebasely launches in beta: Git hosting built for humans and coding agents — handotdev · 2026-10-16
- Drama3 TTS lands in audio.cpp: up to 5x real-time on RTX 5090, <8GB VRAM mode too — Acceptable-Cycle4645 · 2026-10-16
- Three CoreWeave co-founders have sold $3.3B in shares since IPO amid credit worries — SumitGup · 2026-10-16