Cheap tokens, expensive agent: Flash hits $11.97/task vs Luna's $0.05 in Databricks AI Index

krishnan · x · 2026-11-17

Databricks' new AI Index coding-agent benchmark shows low token price doesn't mean low task cost: Gemini 3.8 Flash averaged 140 turns per task at $11.97/task — among the priciest — while GPT-6 Luna and Claude Opus 5.5 both solved 89.1% of tasks at $0.05 and $1.62 per task respectively. Tasks come from real PRs in Databricks' own codebase, scored against hidden tests. Key takeaway: token price is an input, cost per finished task is the bill — Databricks even replayed its own traces before an Opus 5 upgrade and chose to stay on Opus 4.8.

Original post →

More from coding & agent

coding & agent channel →