AI Inference Costs Have Plummeted
BlackHC · x · 2026-07-16
The author rebuts the claim that "AI is the worst technology," providing two core counterarguments:
- For GPT-3.5-level performance, inference costs dropped from $20 per million tokens in November 2022 to $0.07 in October 2024, a roughly 286x decrease
- Epoch estimates suggest that at a fixed capability level, costs are halved approximately every 2 months
They also point out that frontier model parameter sizes haven't just kept growing:
- During the GPT-4 era, sizes reached around 1.8T parameters
- Since then, frontier models have actually shrunk; for example, GPT-4o and Claude 3.5 Sonnet are estimated to have fewer parameters
The conclusion is that efficiency work is indeed happening, a fact ignored by the original article.
Related event: BlackHC Rebuts the “AI Is the Worst Technology” Thesis(5 posts)→
More from Infra
- Cloud capacity crisis is now an AI architecture problem, argues new newsletter — DavidLinthicum · 2026-10-26
- SpaceX files $40B GPU financing commitment with Apollo, already at 1.7GW — XFreeze · 2026-10-26
- Meta plans to raise up to $95B to fund massive AI expansion — Polymarket · 2026-10-26
- Should the community archive open model weights in case Hugging Face gets restricted? — Tall-Ad-7742 · 2026-10-26
- What happens to neocloud GPUs when the AI datacenter debt bubble bursts? — bowdoin-yale · 2026-10-26
- Nebius CEO: Debt Markets Closed for Neoclouds Last Week, Leaning on Customer Advances — zephyr_z9 · 2026-10-26