Litespark serves Nemotron 120B at 962 tokens/sec per user on just two H200 GPUs
NVIDIAAI · x · 2026-10-17
Mindbeam is serving NVIDIA's Nemotron 3.5 Super VL 120B-A12B on the Litespark Inference Engine with strong results: running on just two NVIDIA H200 GPUs, Litespark delivers 962 tokens per second per user and 947 tokens per second of output throughput in its warm-start benchmark. The post notes that state-of-the-art open-source models are gaining traction in academia and industry, driving demand for high-performance inference that maximizes throughput and hardware utilization.
More from Infra
- Texas grid queue hits 474GW; white paper argues AI data centers should become grid resources — espricewright · 2026-10-17
- HF Transformers adds activation checkpoint offloading to CPU to free GPU memory — StasBekman · 2026-10-17
- DatologyAI claims 1000x compute-efficiency headroom from data curation, demo on Oct 27 — josh_wills · 2026-10-17
- Supabase ships MIT-licensed Middleware: write fetch logic once, run on any runtime — dshukertjr · 2026-10-17
- Dev asks for sandbox provider supporting N concurrent sandboxes with dynamically mountable disks — zeeg · 2026-10-17
- SF Signs Data Center Moratorium; Investor Kwok Asks "What Are We Even Doing" — kevinakwok · 2026-10-17