OpenRouter data shows token quality uncorrelated with price — priciest GLM 5.3 provider also breaks more tool calls
remilouf · x · 2026-11-13
remilouf's counterintuitive finding: token quality is not correlated to cost. A scatter plot built on OpenRouter data (week of Nov 4, 2026, providers.dottxt.ai) plots each provider's price vs. the model's median provider against broken-tool-call rates:
- Thousands of providers scatter roughly uniformly — paying more doesn't buy working tool calls
- Outlier: Alibaba is the priciest GLM 5.3 provider (6x median price) while also running 5x the median broken-call rate
- Practical takeaway: price is a useless proxy for reliability when routing across LLM providers; check reliability data directly
Highly relevant for multi-provider routing and agent engineering.
Related event: Token Quality Doesn't Correlate With Price, OpenRouter Data Shows(2 posts)→
More from Models
- Does Qwen3.8-27B survive agent tool-calling loops? Community asks for real-world tests — MajesticPuzzler · 2026-11-13
- Gemini 4 Hydrogen (Flash) spotted rolling out in Antigravity, available to Pro subs — testingcatalog · 2026-11-13
- ChatGPT appears to now render JavaScript when crawling, tests show two very different pages for one URL — gaganghotra_ · 2026-11-13
- Stanford HAI benchmarks: best medical AI agents finish under half of real clinical tasks — StanfordHAI · 2026-11-13
- Figure unveils Handoff²: near-frontier accuracy, completes tasks 2x faster — adcock_brett · 2026-11-13
- Speculation mounts that Nvidia turmoil could delay OpenAI's GPT-6.2 launch — VraserX · 2026-11-13