Don't Just Look at Token Cost for Models
rwang07 · x · 2026-07-11
A recurring point highlighted in a podcast is that when evaluating models, the key isn't the cost per token, but the total cost to solve a given problem.
The post summarizes several guests' perspectives:
- Jason Calacanis solved a problem in 20 minutes using a frontier model that took him 10 hours to solve with open-source models, and at a lower total cost.
- Dylan Patel believes Anthropic's edge lies in token efficiency, not just the per-token price.
- Jensen Huang emphasized that cheap and fast models enable quicker iteration, ultimately leading to better answers.
More from Models
- Moonshot’s Kimi K3 reaches #5 on MathArena as the top open model — xeophon · 2026-07-22
- Google launches Gemini 3.5 Flash Cyber for CodeMender, with limited access for governments — GoogleAI · 2026-07-22
- Kimi K3 tops Gemini 3.6 Flash on four shared public benchmarks — ChrisGPT · 2026-07-22
- Google’s year-long pause in new base-model pretraining draws sharp criticism — teortaxesTex · 2026-07-22
- Current setup is 8,192 input tokens and 2,048 output tokens, with 8k/512 next — TheZachMueller · 2026-07-22
- Kimi K3 feels slower than K2.7, but stronger on long coding jobs and refactoring — Far-Presence2711 · 2026-07-22