Open-source VoxWeft runs local real-time interpreting on Apple Silicon with 124ms TTS latency

面壁智能 · wechat · 2026-10-16

An indie developer open-sourced VoxWeft, a local real-time interpreting system built on ModelBest's VoxCPM2 (2B multilingual TTS) for Apple Silicon — ASR, translation and speech synthesis all run on-device.

Numbers: VoxCPM2 supports 30 languages, 48kHz output and voice cloning, quantized to 4-bit via MLX; first-audio streaming TTS latency of 124–168ms on M3 Ultra, 2.5s end-to-end on M2.

DEVLOG lessons (written as symptom → root cause → fix → cost):

The ASR bridge decoded the 44-byte WAV header as audio samples, injecting loud noise masked by model robustness — caught only in post-open-source code review;
Locking TTS/ASR caused a deadlock: the async generator held the lock while yielding; fix was deleting the lock;
Concurrent model warmup and requests constructed two MLX instances — not thread-safe, process died with exit code -9; asyncio.Lock doesn't work cross-thread, threading.Lock fixed it;
VoxCPM2's temperature scales flow-matching ODE initial noise, not sampling: t=0.01 never emits a stop token and 80% of frames are silent;
7 hand-written samples nearly led to a wrong 'bigger model' conclusion — 28 real ASR clips revealed a config-ordering bug feeding wrong-language text;
Deleting a seemingly redundant stopFlag reset crashed the demo; inherited warmuppatches=2 added 470ms latency and caused the pop it was meant to fix.

The repo also ships AGENT-SETUP.md so an agent can boot the project step by step.

Original post →

Open-source VoxWeft runs local real-time interpreting on Apple Silicon with 124ms TTS latency

More from coding & agent