Fish Drama-3 lands in HF speech-to-speech: voice cloning with inline whisper, laugh and sigh cues
andimarafioti · x · 2026-10-20
Fish Audio's Drama-3 speech model has been merged into Hugging Face's speech-to-speech project via PR, enabling streaming voice conversation. The demo clones the user's own voice, then lets them trigger whispering, laughing, sighing or worried delivery just by talking.
Technical details:
- Integrated via SGLang Omni's speech endpoint, with optional voice cloning from one reference clip plus transcript, multilingual output, and inline emotion, whisper, pacing and vocal cues.
- Reuses the existing HTTP cancellation/teardown path and converts 44.1 kHz audio into the pipeline's 16 kHz blocks with SoXR; no new client dependencies.
- Runs with the official int4 checkpoint: sgl-omni serve --model-path fishaudio/drama-3-int4 --port 8000.
More from coding & agent
- BizAgentBench: new benchmark exposes hidden pitfalls of long-horizon business agents — _akhaliq · 2026-10-20
- AI agent casually builds a wind solver, game flight feels 'freaky' good — MoonL88537 · 2026-10-20
- Runway Launches MCP to Generate Video Directly in ChatGPT, Claude, Grok and Cursor — runwayml · 2026-10-20
- Runway MCP lands in Grok Bot, bringing SOTA media generation into chat — runwayml · 2026-10-20
- Dream Labs launches Wally, a scribe agent that learns your writing style across email, Slack and prompts — Scobleizer · 2026-10-20
- AI agent probes the vibe coding wars: Lovable, not Replit, buys the top 'replit' Google ad — Aiden_Tech_Ai · 2026-10-20