Databricks research team now runs experiments with an agent swarm, training volume up 8x in a year
matei_zaharia · x · 2026-10-17
Ivan Zhou of Databricks says his research team has fully shifted from manually running experiments to a swarm of agents doing the work, with training workloads up over 800% year-over-year.
The key enabler is AIR, Databricks' autoscaling GPU infrastructure: agents can provision GPU endpoints for models from <1B to 1T+ parameters and monitor training on their own. Researchers now focus on defining clear goals and reviewing outcomes, yielding faster model releases, higher quality, and quicker product launches.
AIR is now available to all Databricks customers — the team dogfoods its own product.
More from coding & agent
- AI Agent Delivers Thousands of Pizzas End-to-End via 203,500 Browser Sessions — GaryMarcus · 2026-10-17
- Production-grade RAG assistant: hybrid search, RRF, parent-child chunking, Promptfoo evals — Alternative_Yak_1367 · 2026-10-17
- Tip: append "use a workflow if helpful, your call" to large Claude Code requests — AaronBergman18 · 2026-10-17
- Oracle Engineer Details Full AML Agent Stack: Memory Layers, Skill Distillation, Continual Learning — AI Engineer · 2026-10-17
- Dev ships AI-coded Steam game Nekomancer with a few hundred prompts, 1,500 commits — ssh4net · 2026-10-17
- Super Trouper v0.5.0 ships Pattern tools for Frida-powered MCP reverse engineering workflows — neathack · 2026-10-17