Nemotron 3.5 Lightning Runs on a Single A100, Processing 100 Medical Records for 2 Cents

A developer benchmarked Nvidia's Nemotron 3.5 Lightning (30B total params, 3B active, BF16) locally on a single A100 80GB with vLLM, achieving 169 tok/s and processing 100 discharge records in 34 seconds for about 2 cents, with full reproduction details later published.

2026-10-17 ~ 2026-10-17 · 3 related posts

• Local Nemotron 3.5 on one A100 hits 169 tok/s, processes 100 clinical notes for 2.4 cents — MaziyarPanahi · 2026-10-17
• Benchmark details revealed: 30B-A3B BF16 on one A100 with vLLM 0.27.1 — MaziyarPanahi · 2026-10-17
• Local nemotron 3.5 lightning on one GPU hits 169 tok/s, 100 discharge notes in 34s — MaziyarPanahi · 2026-10-17