Baseten squeezes 7x throughput from Qwen3-ASR, transcribes 1hr audio in under 5s
baseten · x · 2026-10-29
Baseten's voice team shares a deep dive on optimizing Qwen3-ASR into the fastest, cheapest way to run batch transcription:
- Speed: one hour of audio transcribed in under 5 seconds; 7x the throughput of stock vLLM on a single GPU
- Cost: about a tenth of a cent per hour of audio
- Key techniques: native audio decoding, KV cache quantization, and eliminating unnecessary CPU–GPU syncs
The write-up explains how each optimization works and where the gains came from.
Related event: Baseten optimizes Qwen3-ASR: 1 hour of audio transcribed in 5 seconds(2 posts)→
More from Infra
- Marvell said to near megadeal for Lumentum as AI optics race heats up — zephyr_z9 · 2026-10-29
- Engineer stabilizes 64GB DDR3 on dual Asus Z9PE-D8 WS, targeting 256GB for cheap local LLM rigs — richgel999 · 2026-10-29
- Baseten makes Qwen3-ASR transcribe hour-long audio in 5 seconds at ~$0.0013/hr — baseten · 2026-10-29
- Google DeepMind engineer shows five CUDA optimizations that speed up 3D Gaussian Splatting by 10.5x — janusch_patas · 2026-10-29
- Snowflake CEO joins the 'calling top' chorus as analysts flag AI's compute leverage risk — SumitGup · 2026-10-29
- Dave White: a US-China AI training slowdown deal is largely enforceable via zk proofs — _Dave__White_ · 2026-10-29