PyTorch details 2.12x faster on-device streaming ASR on Arm SME2 via ExecuTorch
PyTorch · x · 2026-10-27
PyTorch's official blog, written by Arm engineers, covers porting NVIDIA Nemotron 3.5 streaming ASR to ExecuTorch and profiling it on a Pixel 11.
- Key insight: Qwen3-ASR-1.7B had the best batch accuracy but its WER rose 77% when evaluated on 2.4-second chunks — accuracy must be measured at deployment chunk cadence
- NVIDIA Nemotron Speech Streaming 0.6B achieved the lowest streaming WER at 560ms chunks
- Arm SME2 matrix instructions with FP16 and per-channel INT8 (QC8W) deliver up to 2.12× faster processing on Pixel 11
- Full pipeline detailed: PyTorch model → ExecuTorch export → .pte files → XNNPACK/KleidiAI microkernels → SME2 hardware, plus operator-level profiling
More from Infra
- Anthropic S-1 reveals $800B+ in off-books future compute commitments — GaryMarcus · 2026-10-27
- ~42% of Anthropic's ~11GW contracted compute is NVIDIA GPUs — ns123abc · 2026-10-27
- EdgeRock-4B holds KV cache at 72 MiB across 256K context, 99.1% smaller than Qwen3.5-4B — alifcoder · 2026-10-27
- RTX 5080 price jumps from $1658 to $2318 in ten days on Amazon — Frizzy-MacDrizzle · 2026-10-27
- ComfyUI on 12GB VRAM: measured peaks show offloading turns 24s into 11 minutes — Impossible-Mail-4619 · 2026-10-27
- Neon rebuilds its docs search entirely on its own stack: Lakebase Search, Functions, AI Gateway — matei_zaharia · 2026-10-27