Databricks AI Index: per-token price is a bad proxy — agent task costs vary 32x at equal success rate
CautiousUse8597 · reddit · 2026-11-17
Databricks launched AI Index, built from anonymized usage of 10,000+ enterprises routing through Unity Gateway, plus a coding agent benchmark on real internal tasks. Highlights: Gemini 3.8 Flash's low token price still yields $11.97/task due to 140 average turns; GPT-6 Luna and Claude Opus 5.5 tie at 89.1% success but cost $0.05 vs $1.62 per task (32x apart); Claude Fable 5.1 leads at 96.0% but costs 5x more than Muse Wonder for +1.1 points, and its token share collapsed from 5.5% to 0.6% in six weeks. Single-provider setups fell from 64% to 28% since July; open-weight models rose from 7.3% to 30.9% of token traffic. Median weekly spend per user is $2.49 vs $699.74 at p99. Databricks decides model upgrades by replaying its own traces — that's how it skipped Opus 5 and adopted Opus 5.5 only when it was 29% cheaper per session.
More from coding & agent
- OunceMCP 1.0 RC: Token Budgets and Safe Writes for MCP Servers, 58-85% Fewer Tokens — gentleGiant96 · 2026-11-17
- Solo Dev Shares 6 Hard-Won Lessons on Prompting Video With a Real Person's Face — pvdyck · 2026-11-17
- Who Can Even Read AI-Generated Code? Garbled Screenshot Sparks Debate — gethackteam · 2026-11-17
- Orion foundation model predicts gaze: finds 2x UX bugs, cuts computer-use latency 31% — Scobleizer · 2026-11-17
- Coding Agents May Deliver 100-1000% Productivity Shock to Software Next Year — moultano · 2026-11-17
- Alibaba launches Qwen4-Max: 1M context MoE model tops Chinese models on RSI-Exam — HuaxiuYaoML · 2026-11-17