Debate: is tokens/sec per watt the right efficiency metric for AI?
0xF2 · x · 2026-10-17
0xF2 proposes "tokens/second per watt" as a more intuitive efficiency metric for models. rickasaurus counters that dumber models use far more tokens for the same task, so raw token counts mislead. 0xF2 then suggests a combined unit — a "cogitator": watts needed to output one token/second. A light but interesting thread on how to define AI energy efficiency; no resolution.
More from Infra
- e/acc figure backs local AI as 'civilizational infrastructure' against cloud gatekeeping — beffjezos · 2026-10-17
- GPU prices rose with age: RTX 50 series up 66% YoY, RTX 5090 up 132% on memory shortage — jasonkneen · 2026-10-17
- Huawei details training DeepSeek V4 on Ascend 950 supernodes: 273K tok/s on 2 racks — teortaxesTex · 2026-10-17
- Onchain AI inference surging: Venice hits 500B tokens/day as Bittensor expands subnets — 0xJeff · 2026-10-17
- Anthropic reportedly offers Blackstone 10%+ returns to fund AI chip deal — Polymarket · 2026-10-17
- Nvidia/Lumentum CW DFB laser array shown alongside Claude's rendered version — jwt0625 · 2026-10-17