Ternary quantization shrinks VoxCPM2 5.3x to 827MB, runs real-time on laptop CPU

oguzeldereli · reddit · 2026-11-18

Developer oguzeldereli converted the 2B open-source TTS model VoxCPM2 from bf16 (4.96GB) to ternary 1.58-bit weights (-1/0/+1) with a Q8 diffusion transformer, yielding an 827MB model (+100MB VAE) — 5.3x smaller — served by a custom Python-free C++ runtime.

Quality largely holds:

Speed: RTF 0.16 on RTX 4070 Ti SUPER; real-time on a laptop i9-14900HX CPU only; RTF 2.5 on older CPUs without VNNI. Model (Apache-2.0), runtime and conversion code are all open-sourced. The same TTF method was previously applied to LLM training/memory reduction.

Original post →

More from Multimodal

Multimodal channel →