Cheap tokens, expensive agent: Flash hits $11.97/task vs Luna's $0.05 in Databricks AI Index
krishnan · x · 2026-11-17
Databricks' new AI Index coding-agent benchmark shows low token price doesn't mean low task cost: Gemini 3.8 Flash averaged 140 turns per task at $11.97/task — among the priciest — while GPT-6 Luna and Claude Opus 5.5 both solved 89.1% of tasks at $0.05 and $1.62 per task respectively. Tasks come from real PRs in Databricks' own codebase, scored against hidden tests. Key takeaway: token price is an input, cost per finished task is the bill — Databricks even replayed its own traces before an Opus 5 upgrade and chose to stay on Opus 4.8.
More from coding & agent
- Cohere Labs-backed FeatureGraph standardizes scientific time-series analysis into a single source of truth — Cohere_Labs · 2026-11-17
- Idea: Magic Effects That Generate Code at Training Time, Run It at Execution — SubtleGradient · 2026-11-17
- Same Weights, 5x Cost Gap: HF Engineer's Agent Harness Workshops at AI Dev NYC — DeepLearningAI · 2026-11-17
- A practical CI prompt: make your tests and builds use every CPU core on the runner — DanielLockyer · 2026-11-17
- The boring work behind successful AI agents: data and process cleanup decide outcomes — bigdata · 2026-11-17
- J.Doe enters private beta: proactive decision cards instead of empty prompt boxes — umesh_ai · 2026-11-17