AI Inference Costs Have Plummeted

BlackHC · x · 2026-07-16

The author rebuts the claim that "AI is the worst technology," providing two core counterarguments:

For GPT-3.5-level performance, inference costs dropped from $20 per million tokens in November 2022 to $0.07 in October 2024, a roughly 286x decrease
Epoch estimates suggest that at a fixed capability level, costs are halved approximately every 2 months

They also point out that frontier model parameter sizes haven't just kept growing:

During the GPT-4 era, sizes reached around 1.8T parameters
Since then, frontier models have actually shrunk; for example, GPT-4o and Claude 3.5 Sonnet are estimated to have fewer parameters

The conclusion is that efficiency work is indeed happening, a fact ignored by the original article.

More from Infra