TTS runs fully in-browser on CPU: 5.5x faster than PyTorch, zero network requests
Appropriate_Many7754 · reddit · 2026-10-21
A developer got Ming-3.0-tts-0.5B running entirely in the browser — no server, no API key, no GPU, zero network requests after load.
- Architecture: a Qwen2-0.5B decoder emits one conditioning vector per 80ms of audio, a small diffusion transformer turns it into an audio latent (10 flow-matching steps with CFG), and a streaming vocoder outputs 44.1kHz audio — autoregressive decoding plus a sampler inside a browser tab.
- Performance: on Apple M4 Pro it speaks 2x real-time, 5.5x faster than the PyTorch reference on the same CPU, starts in 0.3s, and uses 4.5x less memory; full methodology in benchmarks.md.
- Toolkit: fusor-ai (Rust kernels, thread pool, reusable Qwen2/DiT/iSTFT blocks, streaming weight format, browser loader) is on crates.io (MIT), aiming to make the next model port much cheaper.
Demo and code are public; the author is taking suggestions for the next model to port (ASR, small LLM, embeddings).
More from Infra
- Cadence announces sovereign AI cloud for European chip designers — karlfreund · 2026-10-23
- SGLang Deep-Dive Blog Series Breaks Down KV Cache, Posts up to 15.5x Throughput Gains — BanghuaZ · 2026-10-23
- DynaLM: Open-Source CPU-First LLM Inference Engine Seeks Feedback — Unlikely_Safety_7456 · 2026-10-23
- Capital Is Now the AI Buildout Bottleneck, Says Analyst Citing Jensen Huang — JOBhakdi · 2026-10-23
- Cerebras explains how it deploys trillion-parameter models on wafer-scale chips — bookwormengr · 2026-10-23
- IFM open-sources RL360, the agentic RL infra behind K2 Horizon's training — rupspace · 2026-10-23