Actual Computer serves 8,000+ tokens/s from a single NVIDIA DGX Station
cephaloform · x · 2026-10-16
Actual Computer published a case study showing its inference engine 'e' turns an NVIDIA DGX Station into a department-scale AI factory, served via api.actual.inc over confidential compute so even Actual can't see requests.
Key numbers: running NVIDIA Nemotron 3.5 Super VL (FP8, MTP on, 1M context) on one DGX Station delivers 455 tokens/s on a single stream, scaling past 8,000 tokens/s at 128 streams and over 10,000 at 256 streams, with 0.58s first-token latency on a fresh 8K prompt. The team argues DGX Stations will be the local AI workhorse of the 'token factory era.'
Related event: New Inference Engine 'e' Hits 10,000+ tok/s on a Single DGX Station(4 posts)→
More from Infra
- Copper to optical: optical backplanes seen as necessary as racks move to optics — BenBajarin · 2026-10-16
- Polymarket prices 28% odds a US state enacts data center moratorium by year-end — Polymarket · 2026-10-16
- Pennsylvania residents reject $10,000-per-household offer for massive Amazon data center — Polymarket · 2026-10-16
- PyTorch now runs natively on AWS Trainium with zero code changes — PyTorch · 2026-10-16
- Chinese labs run on ~1/10 US compute; ByteDance pays 2-4x to hoover up SEA supply — robleclerc · 2026-10-16
- Zymtrace raises $12.2M and hires ML engineers to build agents that fix GPU bottlenecks — cfregly · 2026-10-16