Databricks Slashes Internal AI Costs by 90% via AI Gateways and Smart Routing
AdiPolak · x · 2026-08-10
Databricks co-founders shared how they drastically reduced internal AI spending (up to 90% in some scenarios) while aggressively growing adoption, treating AI tokens as an optimizable engineering resource.
The core strategy involves centralizing control through their Unity AI Gateway for analysis and smart routing. Key techniques include:
- Shifting Default Models: Routing coding tasks that don't require maximum intelligence to cheaper OSS models like GLM, saving over 50%.
- Smart Routing: Automating model selection to squeeze out further efficiencies.
- Engineer-level Budgets: Pushing control down to engineers to set per-task token budgets, preventing unexpected cost spikes.
More from coding & agent
- DIRECT: retrieval-based next-question suggestions for conversational agents — burkov · 2026-10-16
- Claude Code and Codex now autocomplete the user's next action — tokenbender · 2026-10-16
- Hermes Agent ships official Reverse Engineer Anything plugin with one-command setup — Teknium · 2026-10-16
- MLExpert: 35B MoE Agent Trained to Autonomously Handle Machine Learning Engineering — Shangheng Du · 2026-10-16
- Cowork-Gym: Alibaba Synthesizes 64.9K Agent Work Environments, Boosting Qwen by 8.7 Points — Anni Zou · 2026-10-16
- MMSearch-OPD: Joint RL and On-Policy Distillation for Long-Horizon Multimodal Search Agents — PekingUniversity · 2026-10-16