From Intelligence to Cost-Efficiency: The Evolving Core Metric for LLMs

beffjezos · x · 2026-08-13

Highlighting the progression of the most relevant metrics for evaluating LLMs:

Intelligence: The primary focus during the early stages.
Intelligence per $: The current battleground, focusing on cost-efficiency.
Intelligence per $ per second: The future metric, demanding extreme inference speed on top of low costs.

The author further concludes that asymptotically, intelligence per watt (energy efficiency) is all that ultimately matters.

More from Infra