Databricks evals: Opus 5.5 cuts task costs 20%, GPT-6 Luna is 20x cheaper
pwendell · x · 2026-09-29
Databricks' Peter Wendell shared findings from online workload analysis of 2,400 engineers plus offline evals on recent model releases:
- Opus 5.5 and GPT-6 Luna meaningfully expand the cost/quality Pareto frontier.
- Opus 5.5 is now the highest-quality mid-tier model, beating all prior Opus versions as well as GPT-6 Sol and GPT-5.6 Sol.
- Against Opus 4.8 (the prior cheapest Opus; 5.0 was a dud), Opus 5.5 consistently reduces same-task costs by 20%, and Databricks is now encouraging it as the everyday default for coding.
- GPT-6 Luna is extremely cheap: at least 20x cheaper per task than Opus 5.5 in every offline benchmark and observed online use.
- Luna is surprisingly capable — on one of Databricks' hardest eval suites it roughly matches Opus 4.6 while costing 99.3% less per task (preliminary finding).
Their production setup routes workloads via Unity Gateway, with harnesses including Omingent, Claude Code, Codex, and Cursor.
Related event: GPT-6.1 Sol Launches with Astra-Level Intelligence at a Fifth of the Price(32 posts)→
More from coding & agent
- Generic API-catalog MCP server vs one tool per job: a context-bloat experiment — MandeepKaur9407 · 2026-11-16
- Berkeley paper: LLMs answer with stale values even when the update is in context — ombre-ai · 2026-11-16
- Effective HTML: 4.2k-star agent skills turn wireframes into self-contained interactive prototypes — tom_doerr · 2026-11-16
- skills-gate: one gate skill fronts 148 agent skills at just 389 tokens — Distinct-Pie2389 · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- shadcn boosts @uisdk: one-shot UI rebuild that's instantly E2E-testable by AI — shadcn · 2026-11-16