Databricks Slashes Internal AI Costs by 90% via AI Gateways and Smart Routing
AdiPolak · x · 2026-08-10
Databricks co-founders shared how they drastically reduced internal AI spending (up to 90% in some scenarios) while aggressively growing adoption, treating AI tokens as an optimizable engineering resource.
The core strategy involves centralizing control through their Unity AI Gateway for analysis and smart routing. Key techniques include:
- Shifting Default Models: Routing coding tasks that don't require maximum intelligence to cheaper OSS models like GLM, saving over 50%.
- Smart Routing: Automating model selection to squeeze out further efficiencies.
- Engineer-level Budgets: Pushing control down to engineers to set per-task token budgets, preventing unexpected cost spikes.
More from coding & agent
- Dev jokes: 'It's not vibe coding if you actually care about the code' — haydendevs · 2026-08-10
- Scale AI Founder: Misaligned Multi-Agent Swarms Now Finding 0-Days — alexandr_wang · 2026-08-10
- Engineering Guardrails to Prevent Auto-Reply AI Agents from Infinite Loops — kumard3 · 2026-08-10
- Training AI Coding Agents in Remote Sandboxes with TRL and OpenCode — NielsRogge · 2026-08-10
- 13 open-source frameworks and SDKs for building AI agents — TheTuringPost · 2026-08-10
- Meta Prices Coding Agent Below Cost to Trade for Training Data — shashib · 2026-08-10