Vercel: decision models took 26% of gateway requests for 0.53% of spend
nukebuddy · reddit · 2026-11-15
A detail in Vercel's November AI Gateway production index deserves more attention than the model rankings: in October, decision models — which classify, score, or choose between options rather than write long answers — handled 26% of gateway requests but accounted for just 0.53% of spending, at roughly 1/70th the cost per request of language models.
- About a third of paid teams used one; among teams processing 10B+ tokens a month, it was seven in ten.
- The takeaway: separate decisions from generation. Triaging a ticket and drafting the reply are different jobs, and sending both to the same large model may be convenient but wasteful.
- Caveats: this is Vercel gateway traffic only, not the whole market; spend is estimated at list prices; almost all decision-model volume went to one provider (Jev); and no independent accuracy comparison is given for specific workloads.
Teams routing everything through one model should ask which steps actually need generated text.
More from Infra
- Report: Top 4 Chinese AI Labs Trained New Models on Export-Banned Nvidia Blackwell Chips — Traditional-Chip8339 · 2026-11-15
- Alibaba's T-Head files for Hong Kong IPO, spinning off data-center chip business — pstAsiatech · 2026-11-15
- Bitcoin miner MARA to sell large amounts of BTC to fund AI data center expansion — Polymarket · 2026-11-15
- Epoch AI chart: AMD MI525X doubles Nvidia GB300 inference performance per chip — AccBalanced · 2026-11-15
- J.P. Morgan: Nvidia's share of future frontier lab compute deals falls to 39% from 80%+ — Gregory_C_Allen · 2026-11-15
- Dev rewrites OpenCode Browse in Rust for $60, cutting memory use by orders of magnitude — AccBalanced · 2026-11-15