Ultrafast Qwen3-TTS: 34ms TTFA & 10 RPS on H100
Forsaken_Goal3692 · reddit · 2026-08-22
Nari Labs open-sourced a high-speed implementation of Qwen3-TTS 1.7B, addressing slow response times in local models. On a single H100, it achieves 10 RPS with a sub-50ms p95 Time-to-First-Audio (TTFA), scaling to 20 RPS at sub-100ms p95 TTFA. A 4090 can achieve 50ms p95 TTFA with tuning. Benchmarks show significant speedups over vLLM-Omni and SGLang-Omni.
More from Infra
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16
- vLLM ships Day-0 LoRA RL for Muse Spark Open, lifting AIME from 0.56 to 0.85 — vllm_project · 2026-10-16
- DSpark draft model for Muse Spark Open proposes 7 tokens per pass with 23-42% acceptance — vllm_project · 2026-10-16
- Cloudflare Stream's WebRTC ultra-low-latency live streaming is now Generally Available — ritakozlov · 2026-10-16
- YC-backed Overshoot launches inference optimization, claims $5M saved for AI teams — ycombinator · 2026-10-16