ExecuTorch's New WebGPU Backend Runs Whisper and Vision Models In-Browser, 4-10x Faster
shoumikgoswami · reddit · 2026-11-11
PyTorch's new ExecuTorch WebGPU backend runs more than chat in the browser: validated models include Whisper (speech-to-text), SmolVLM (image QA), segmentation and object detection — local input processing without shipping recordings or images to a server.
Deployment: export to an ExecuTorch .pte file → serve with a web page → run via WebAssembly + WebGPU. The same artifact also runs natively on Metal, Vulkan and D3D12.
Performance: vs its own multithreaded WASM/XNNPACK CPU path, 4-7x faster prefill and 2-10x faster decode (Chrome on Apple M1 Pro, quantized Llama 3.2 1B and Qwen3-0.6B; caveat: fp32 on WebGPU vs dynamic 8-bit on CPU, and no comparison against cloud models).
The author's take: moving an assistant's speech/vision input processing onto the user's device may matter more than squeezing in another small chat model. The backend is still early.
Related event: PyTorch ExecuTorch Launches WebGPU Backend for In-Browser Models(4 posts)→
More from Infra
- Batch APIs: the 50% LLM discount only pays off after model choice and prompt caching — arpit_bhayani · 2026-11-13
- RTX Pro 6000 bricks mid-Qwen run, owner jokes it "died of a broken heart" over Jensen news — wine_dark · 2026-11-13
- Bittensor's Chutes Handles 256 Concurrent Requests on a Single RTX 5090 at ~18k tok/s — markjeffrey · 2026-11-13
- Google expands TPU v8 lineup: 8tx cuts latency 2x, 8i Max doubles silicon per rack — Beth_Kindig · 2026-11-13
- Horace He to share two mental models for reasoning about inference at SGLang Summit — cHHillee · 2026-11-13
- Used A100 prices fall another 20-30% as sellers rush ahead of a collapse — Bascilian · 2026-11-13