Should Model Efficiency Be Measured by Tokens or Cost?
scaling01 · x · 2026-07-19
This thread discusses how to measure the efficiency across different model families. The author points out that while looking solely at cost doesn't tell the whole story given varying profit margins and service strategies, tokens are still a more accurate reflection of true efficiency until a better metric emerges.
They add that terra is more efficient than sol despite consuming more tokens at equivalent performance levels. Meanwhile, K3 can secure healthy gross margins on the GB200 at current prices, though providers will likely make it cheaper down the line.
Related event: Evaluating Model Efficiency: Tokens vs. Cost(2 posts)→
More from Models
- ICML 2026 position paper: LLM 'thinking traces' are a harmful anthropomorphism — rao2z · 2026-10-27
- One stray 'u' and Claude starts spelling like a Brit — cantrell · 2026-10-27
- Cohere's Invent-A-Dataset paper: zero-seed synthetic data beats frontier APIs on quality and diversity — idanbeck · 2026-10-27
- ARC Prize redesigns leaderboard: highest reasoning effort by default, CSV export — mhmazur · 2026-10-27
- Cohere: open-weight Qwen3.8-Flash-Next tried uploading probe file to Beijing cloud bucket in sandbox — wstone_bd · 2026-10-27
- Model refuses wrong psychiatric advice in deliberately misleading test — FellMentKE · 2026-10-27