Matei Zaharia's arc: from Apache Spark to the $190B Databricks and reliable AI agents
jfiance · x · 2026-10-02
A profile traces Matei Zaharia's Berkeley story:
- Spark origins: He started Apache Spark as a PhD project in 2009 in the lab that became AMPLab; it grew into one of the world's most-used big-data systems and laid the foundation for Databricks, now valued at $190B.
- Sustained open source: He also led MLflow and Delta Lake, and received the 2025 ACM Prize in Computing for distributed data systems — bookending his Spark dissertation winning the 2014 ACM Doctoral Dissertation Award.
- New chapter: Still Databricks co-founder/CTO, he's back at Berkeley EECS as an associate professor researching how to build and scale reliable agents, co-authoring DSPy and GEPA.
- The author frames it as Berkeley's startup flywheel: campus invention → global open-source adoption → category-leading company → founders returning to mentor the next generation.
Related event: From Spark to Databricks: Matei Zaharia's Berkeley Startup Journey(2 posts)→
More from Companies & People
- Sentry CEO funded mitsuhiko to bring MCP into Pi — zeeg · 2026-10-02
- Sentry CEO teases badlogic over his move: 'look at this sellout' — zeeg · 2026-10-02
- Grok 4.7 now available on the Gemini Enterprise Agent Platform — SpaceXAI · 2026-10-02
- UChicago CS hiring tenure-track faculty in robotics, embodied AI — plopesresearch · 2026-10-02
- Seroter: AI agents are making app frontends optional—developers should build CLIs, skills and MCP servers — rseroter · 2026-10-02
- Exclusive: Google expands pilot paying developers and small businesses for proprietary offline code — sourdub · 2026-10-02