Qualcomm's NPU runs Qwen3 1.7B locally at 48.2 tok/s at half Nvidia's price
draginol · x · 2026-10-19
draginol points out that amid all the Nvidia attention, Qualcomm's NPU quietly delivers impressive small-model performance at half the price. A screenshot shows a Snapdragon X2 Elite (Adreno X2-90 GPU, 48GB RAM) running Qwen3 1.7B W4A16 locally via Genie QNN: 3,810 tok/s prefill at 1K context, 48.2 tok/s decode, 301ms first token, and 70.0% accuracy (56/80 items) — evidence that on-device NPUs are already viable for local LLM inference.
More from Infra
- IBM exec: US AI leadership hinges on permitted power and domestic robotics scale vs China — ingliguori · 2026-10-19
- Matthew Green: What if all frontier AI models go down in a major incident? — matthew_d_green · 2026-10-19
- HotKV semantic cache cuts Claude tokens 70%, but answer quality dips until threshold 0.88 — HotKV-Ltd · 2026-10-19
- Critics question Anthropic compute shortage amid virtual 'scientists' claims — gerardsans · 2026-10-19
- David Sacks: payback period is the North Star metric for AI infra — CoreWeave recaps in 2.5 years on 5-year deals — DavidSacks · 2026-10-19
- Qwen3.8 27B benchmarks on NVIDIA RTX Spark: 19.3 tok/s decode, 100% accuracy — draginol · 2026-10-19