Chutes' Parallax hits ~18,000 tok/s on one RTX 5090, ~1 cent per million tokens
bittingthembits · x · 2026-11-13
Chutes (Bittensor SN64) says its Parallax setup runs at nearly 18,000 tokens/sec on a single $2K RTX 5090, serving 256 concurrent requests on a custom vLLM fork — faster than Llama 3.1 8B on a datacenter H100 with standard vLLM (3,500 tok/s) or even SGLang max batch (16,215 tok/s).
Per the chart, GPU-rental cost works out to $0.012 per million tokens (about 1 cent), vs $0.051–0.074 for H100 setups. Caveats: Parallax is a small MoE with 4-bit expert layers doing less math per token than a dense 8B, answer quality is unbenchmarked, and it's an informal benchmark with differing test setups.
More from Infra
- H100 rental fell from ~$8/hr in 2023 to ~$2.50 as decentralized training net SN44 keeps running — richdotca · 2026-11-13
- Unverified: Jensen Huang steps down as NVIDIA CEO; traders favor CUDA creator Ian Buck at 34% — morganlai · 2026-11-13
- Chinese major models reportedly trained on Blackwell-class chips, though scale disappoints — teortaxesTex · 2026-11-13
- Sentdex: 'China spy' claim is likely BS — more likely mutual export-restriction circumvention — Sentdex · 2026-11-13
- Analyst: DeepSeek V4's Flash variant is winning, and a hyper-efficient 2B V5 could be the real play — teortaxesTex · 2026-11-13
- vLLM ships recipe for Thinking Machines' Inkling-2-Small: 594B MoE, 1M context, ~1,100 tok/s/GPU — vllm_project · 2026-11-13