Fish Audio S2 fork jumps from 1.2 to 23 t/s on Apple Silicon

fogonthebarrow-downs · reddit · 2026-07-23

A developer forked Fish Audio’s S2 and optimized it for Apple Silicon, claiming a huge speedup on MPS.

• Through quantization and runtime changes, throughput reportedly improved from about 1.2 tokens/s to 12.4 t/s in bf16 and 23 t/s in int8.
• The gains came mostly from KV-cache reuse, GQA expansion, an int8 kernel for MPS, Torch compile on Metal/Inductor, and sub-batch streaming.
• The benchmark was run on an M5 Pro 48GB, and the author is asking other Apple Silicon users to reproduce it on different chips.
• This is a useful local TTS optimization post for people working on Apple-device inference.

Original post →

More from Multimodal

• A multimodal “task” prompt for agents bundles voice, screen, marks, and text — omarsar0 · 2026-07-23
• Replicate Designer MCP adds Flux 1.1 Pro image generation to tool workflows — modelcontextprotocol · 2026-07-23
• fal and Sequoia Host the World's First Generative Video Hackathon — Jamez_Chi · 2026-07-23
• A 3D blockout plus phone walkthrough becomes a video-generation workflow — Bart_Bialobrzeski · 2026-07-23
• Clean Plate LoRA examples show a practical video-cleanup workflow — nazihater3000 · 2026-07-23
• Qwen-image 3.0 is now available on Runware — aziz4ai · 2026-07-23

Multimodal channel →