Same Cloud Machine Size, Nearly 2x Speed Gap: A Speech Recognition CPU Lottery Postmortem
shauntrennery · x · 2026-11-13
South African radio transcription service locl found that identical Fly performance-1x machines decoded 30-second audio windows in 4.1-4.4s or 7.5-7.8s — nearly a 2x gap. Root cause: AMD EPYC spans multiple CPU generations, and the cloud dashboard doesn't distinguish them. Since 4.2s/window keeps pace on one core while 8s (107% utilization) falls permanently behind, they adopted a gate: any new speech-recognition machine must decode three real windows before taking traffic, and is destroyed if they take more than 6 seconds. A practical postmortem for anyone doing single-core inference deployment on cloud VMs.
More from Infra
- Bolt Graphics unveils Zeus GPU with expandable memory up to 384GB, mass production in 2027 — HankYeomans · 2026-11-13
- Open OCR costs fell 5x in 18 months: $0.51 to $0.095 per 1,000 pages, benchmarked on 850 pages — vanstriendaniel · 2026-11-13
- Linus Ekenstam: China's energy dominance will 'blow everything else out of the water' — LinusEkenstam · 2026-11-13
- Nscale admits no binding commitments for $32B Anthropic megaproject as losses hit $4 per $1 revenue — GaryMarcus · 2026-11-13
- Jensen Huang admitted facing an 'avalanche' of excess wafers and memory amid GPU/DRAM crisis — oscarfr · 2026-11-13
- Back-of-envelope: serving 1B AI Mode queries on Gemini costs only ~$4M — gaganghotra_ · 2026-11-13