Qualcomm demos an on-device decision model routing AI requests in 8.6ms on NPU
carrycooldude · x · 2026-11-17
A Qualcomm dev account highlights Lucas da Silva's experiment with open-source Laya: a small decision model that inspects every AI request first, running on the Snapdragon X Elite NPU with no quantization or retraining. It detects private data, triages requests, and decides whether work stays local or goes to the cloud.
Results: 8.6ms per decision vs 36ms on CPU, with 20x+ less energy — a promising pattern for local-first routing before bigger models step in.
More from coding & agent
- Codex lacks visual indicator for multi-day agent runs, user resorts to renaming tasks — chrisalbon · 2026-11-17
- Poolside's 'everything collapses into the model' thesis shapes new agent runtime Rex — sull · 2026-11-17
- FrontierCS 2 benchmark challenges AI agents to beat published CS research blind — arena · 2026-11-17
- VPS MCP Server lets AI models manage Docker containers via Portainer API — modelcontextprotocol · 2026-11-17
- User's open letter to OpenAI: Dot agent degraded after a week, switching to Claude Duet — Danny7Guns · 2026-11-17
- Developer outlines quest to bring Three.js rendering quality up to AAA game engines — anselm · 2026-11-17