Ultrafast Qwen3-TTS: 34ms TTFA & 10 RPS on H100
Forsaken_Goal3692 · reddit · 2026-08-22
Nari Labs open-sourced a high-speed implementation of Qwen3-TTS 1.7B, addressing slow response times in local models. On a single H100, it achieves 10 RPS with a sub-50ms p95 Time-to-First-Audio (TTFA), scaling to 20 RPS at sub-100ms p95 TTFA. A 4090 can achieve 50ms p95 TTFA with tuning. Benchmarks show significant speedups over vLLM-Omni and SGLang-Omni.
More from Infra
- Running DeepSeek-V4-Flash on 64GB Mac: Small cache beats large — cowboy-bebob · 2026-08-22
- Request: Best setup for Qwen 3.8 27B on RTX 6000 Blackwell — ApolloPS2 · 2026-08-22
- Supermicro Investigation Clears CEO in $2.5 Billion Nvidia Smuggling Scheme — fortune · 2026-08-22
- Qwen3.8-27B: 3x Faster Long-Context Decoding with DFlash2 + XQA — stargate425 · 2026-08-22
- Monad Community Touts a New Era of 100ms Latency — tomwanhh · 2026-08-22
- Intel Arc Pro B70 Benchmarks: 52 tok/s with Qwen3.8-27B via vLLM XPU — No-Dot-Not · 2026-08-22