Grok 4.5 Auto-Optimizes Speech-to-Text
Nilofer_tweets · x · 2026-07-15
Someone ran an automated research loop using Grok 4.5 to make Parakeet TDT 0.6B v3 run faster on CPUs.
The results:
- Identified that the encoder takes up about 98% of the runtime, so experiments focused there.
- Switching the encoder from the standard Hub dynamic INT8 path to static quant yielded significant performance gains.
- On an AMD EPYC machine with 8 vCPUs, throughput increased from roughly 26x real-time to 54x real-time, a 2.1x speedup.
- On Mac, long audio scenarios also saw a 1.4x speedup.
More from coding & agent
- Agent verified 'success' after double-voting: a lesson in harness vs model bugs — TejasKumar_ · 2026-10-26
- diginomica: agentic automation hype is overdone, and 'The Hackening' of enterprise security has begun — jonerp · 2026-10-26
- 'Vibe verification': ~60% of AI-written formal specs prove the wrong property — basedjensen · 2026-10-26
- Devin launches Security suite powered by Claude Mythos with $100M+ credits — DevinAI · 2026-10-26
- HackBench injects reward hacks into agent trajectories to stress-test monitors — DanielKhashabi · 2026-10-26
- Hugging Face ships hf CLI updates: deduped cache saves 19GB, faster buckets, retryable GPU jobs — yb2698 · 2026-10-26