Token Prices Fell 99% in 2.5 Years Yet Total Spend Rose; H100 Spot Climbed All Along
le_james94 · x · 2026-09-26
Lecture 2 of Stanford's MS&E 435:
- Cost per token fell 99% over 2.5 years, yet total spend went up.
- Reasoning models and agents burn orders of magnitude more tokens per task, and H100 spot prices climbed through the entire decline.
- Takeaway: a fast chip is not a business — demand inflation eats the price decline.
More from Infra
- Altman says OpenAI aims to replace Nvidia in the medium to long term — zephyr_z9 · 2026-11-11
- Chinese University Bought Nvidia H200 Chips Despite US Blacklist — pstAsiatech · 2026-11-11
- Baidu's ERNIE Evolve tops public bests on NanoGPT Speedrun, brings RSI to real inference optimization — xiaohu · 2026-11-11
- Baidu says RSI auto-optimized ERNIE inference, 6.24x decode speedup on Kunxin P800 — xiaohu · 2026-11-11
- Nvidia's supply commitments jump from $119B to $279B in one quarter, locking up 2027-28 memory — tengyanAI · 2026-11-11
- Train 27B models on a 24GB GPU: ternary optimizer cuts memory to 1 byte/weight — oguzeldereli · 2026-11-11