Stanford's Intelligence per Watt: hybrid local-cloud routing cuts cost 60-80%
rohanpaul_ai · x · 2026-09-11
A new Stanford/Together AI paper (authors incl. Christopher Ré, John Hennessy) proposes Intelligence per Watt (IPW) — task accuracy per unit of power — as a unified metric for local LLM inference.
- Evaluated 20+ local SOTA models (≤20B active params), 8 accelerators, and 1M real-world chat/reasoning queries.
- Hybrid local-cloud routing cut energy, compute, and cost by 60–80% vs a batched-cloud baseline.
- From 2023 to 2025, local AI's intelligence-per-watt improved 5.3×; locally serviceable query coverage jumped from 23.2% to 71%+.
The takeaway: small local models plus laptop-class accelerators (e.g. Apple M4 Max) can meaningfully redistribute demand away from centralized cloud infrastructure.
Related event: Stanford: Hybrid Local-Cloud Inference Cuts 60-80% of Compute Cost(3 posts)→
More from Infra
- MySQL recursive CTEs time out at 500M rows; C# BFS answers in 2.7ms vs NebulaGraph 23.6ms — JeremyCMorgan · 2026-10-28
- Token demand on OpenRouter grew 27x to 165T weekly, open models now serve 72% — FinanceYF5 · 2026-10-28
- AI's unusual bet: buying compute for years, selling intelligence by the token — vaibhavbetter · 2026-10-28
- OpenAI's Will Depue says compute locks in US primacy as 'end of human history' nears — willdepue · 2026-10-28
- Malaysia has overtaken Singapore in data center capacity, planned buildout 10x — bytebot · 2026-10-28
- Why Cloudflare Is the Best Platform to Build AI Agent Operating Systems — ritakozlov · 2026-10-28