Fish Audio S2 fork jumps from 1.2 to 23 t/s on Apple Silicon

fogonthebarrow-downs · reddit · 2026-07-23

A developer forked Fish Audio’s S2 and optimized it for Apple Silicon, claiming a huge speedup on MPS.

Through quantization and runtime changes, throughput reportedly improved from about 1.2 tokens/s to 12.4 t/s in bf16 and 23 t/s in int8.
The gains came mostly from KV-cache reuse, GQA expansion, an int8 kernel for MPS, Torch compile on Metal/Inductor, and sub-batch streaming.
The benchmark was run on an M5 Pro 48GB, and the author is asking other Apple Silicon users to reproduce it on different chips.
This is a useful local TTS optimization post for people working on Apple-device inference.

More from Multimodal