Matei Zaharia

Created the Apache Spark distributed computing framework during his PhD at UC Berkeley's AMPLab in 2009. Spark introduced Resilient Distributed Datasets (RDDs) for fault-tolerant in-memory cluster computing, becoming the most widely used engine for large-scale data processing, with over 2,000 contributors and adoption across virtually every Fortune 500 company.

Co-founded Databricks in 2013 and serves as CTO, building the Lakehouse platform that unifies data warehousing and data lakes. The company reached a $134B valuation by late 2025 with $4.8B+ annual revenue run-rate, becoming the leading enterprise data and AI platform.

Created MLflow in 2018 as an open-source platform for managing the complete machine learning lifecycle -- experiment tracking, reproducibility, and model deployment. MLflow is now the most popular ML lifecycle management tool, with tens of thousands of users.

Co-created Delta Lake, an open-source ACID transaction layer for cloud data lakes, enabling reliable data engineering at scale. Published at VLDB 2020, Delta Lake is a cornerstone of the Lakehouse architecture.

Authored the influential 2024 BAIR blog post 'The Shift from Models to Compound AI Systems,' arguing that state-of-the-art AI results increasingly come from multi-component systems rather than monolithic models, shaping industry direction toward retrieval-augmented and agent-based architectures.

Co-developed DSPy, a framework for programming (rather than prompting) language models, enabling declarative composition of LLM pipelines that self-optimize. Accepted at ICLR 2024.

Co-developed ColBERT (and ColBERTv2), a late-interaction neural retrieval model that achieves state-of-the-art passage search quality with practical efficiency. Published at NAACL 2022.