Databricks AI Index: per-token price is a bad proxy — agent task costs vary 32x at equal success rate

CautiousUse8597 · reddit · 2026-11-17

Databricks launched AI Index, built from anonymized usage of 10,000+ enterprises routing through Unity Gateway, plus a coding agent benchmark on real internal tasks. Highlights: Gemini 3.8 Flash's low token price still yields $11.97/task due to 140 average turns; GPT-6 Luna and Claude Opus 5.5 tie at 89.1% success but cost $0.05 vs $1.62 per task (32x apart); Claude Fable 5.1 leads at 96.0% but costs 5x more than Muse Wonder for +1.1 points, and its token share collapsed from 5.5% to 0.6% in six weeks. Single-provider setups fell from 64% to 28% since July; open-weight models rose from 7.3% to 30.9% of token traffic. Median weekly spend per user is $2.49 vs $699.74 at p99. Databricks decides model upgrades by replaying its own traces — that's how it skipped Opus 5 and adopted Opus 5.5 only when it was 29% cheaper per session.

Related event: Databricks AI Index: open-weight models surge to ~30% of enterprise tokens as 75% of firms go multi-vendor(9 posts)→

Original post →

More from coding & agent

coding & agent channel →