AI Inference Costs Have Plummeted
BlackHC · x · 2026-07-16
The author rebuts the claim that "AI is the worst technology," providing two core counterarguments:
- For GPT-3.5-level performance, inference costs dropped from $20 per million tokens in November 2022 to $0.07 in October 2024, a roughly 286x decrease
- Epoch estimates suggest that at a fixed capability level, costs are halved approximately every 2 months
They also point out that frontier model parameter sizes haven't just kept growing:
- During the GPT-4 era, sizes reached around 1.8T parameters
- Since then, frontier models have actually shrunk; for example, GPT-4o and Claude 3.5 Sonnet are estimated to have fewer parameters
The conclusion is that efficiency work is indeed happening, a fact ignored by the original article.
Related event: BlackHC Rebuts the “AI Is the Worst Technology” Thesis(5 posts)→
More from Infra
- Tabul AI launches Metal TreeSHAP to speed up Shapley values on Apple silicon — Scobleizer · 2026-07-22
- DeepSeek-V4-Flash tops out at 770 tok/s on one B300 in a vLLM batch test — Moreh · 2026-07-22
- NVIDIA starts shipping 102.4 Tbps Spectrum-6 switches for Vera Rubin AI factories — nvidia · 2026-07-22
- Apple publishes SOC 3 audit reports for Private Cloud Compute — throwfaraway4 · 2026-07-22
- Reddit GPU renters say existing platforms only give you two of three: code, recovery, fair billing — legendpizzasenpai · 2026-07-22
- The Sandboxing Manifesto: Secure Execution Environments for Agents — spirosoik · 2026-07-22