Grok 4.5 Auto-Optimizes Speech-to-Text
Nilofer_tweets · x · 2026-07-15
Someone ran an automated research loop using Grok 4.5 to make Parakeet TDT 0.6B v3 run faster on CPUs.
The results:
- Identified that the encoder takes up about 98% of the runtime, so experiments focused there.
- Switching the encoder from the standard Hub dynamic INT8 path to static quant yielded significant performance gains.
- On an AMD EPYC machine with 8 vCPUs, throughput increased from roughly 26x real-time to 54x real-time, a 2.1x speedup.
- On Mac, long audio scenarios also saw a 1.4x speedup.
More from coding & agent
- Coding agents are heading toward an AI-writes, AI-reviews, human-approves workflow — aftahi_ai · 2026-07-22
- oMLX 0.5.2 adds Mac menu-bar stats, low-bit decode kernels, and faster downloads — awnihannun · 2026-07-22
- GitHub review bot hits its PR limit and forces a 39-minute cooldown — DanielLockyer · 2026-07-22
- Max reasoning effort appears to be mobile-only in Codex Remote, not desktop — GabGarrett · 2026-07-22
- A Reddit demo argues online stores should expose carts and pricing through MCP — gelembjuk · 2026-07-22
- Open-source AI SDK provider routes Vercel apps through a local Codex subscription — lgrammel · 2026-07-22