Data Lakehouse

A data lakehouse combines the low cost and flexibility of a data lake with the reliability and performance of a data warehouse. It stores data in open file formats on cheap object storage but adds a transactional table layer that provides ACID guarantees, schema enforcement, time travel, and SQL performance — capabilities once exclusive to warehouses. The result is one platform serving BI, data science, and machine learning without copying data between separate systems.

How It Works

The foundation is object storage (S3, ADLS, GCS) holding columnar files such as Parquet. On top sits an open table format — Delta Lake, Apache Iceberg, or Apache Hudi — that maintains a transaction log and metadata, enabling atomic commits, concurrent writes, schema evolution, partition evolution, and snapshot/time-travel queries. Compute engines (Spark, Trino, Flink, Databricks SQL, Snowflake) read and write these tables independently of storage, so storage and compute scale separately. A catalog (Unity Catalog, AWS Glue, Polaris) governs tables, lineage, and access.

When to Use It

Choose a lakehouse when you want a single source of truth for both analytics and ML, when datasets are large and varied (structured and semi-structured), and when avoiding lock-in via open formats matters. It is well suited to organizations consolidating separate lake and warehouse stacks to cut data duplication and ETL sprawl.

Trade-offs

Lakehouses are newer and operationally more complex than turnkey warehouses; tuning file sizes, compaction, and clustering is necessary for good performance. Concurrent-write conflict handling and metadata scaling require care on very high-throughput tables. For small, purely relational BI workloads, a managed warehouse may be simpler. Governance and small-file management need active attention, and feature parity across engines and table formats varies.

Related Patterns

The lakehouse is typically organized with medallion-architecture, exposes curated marts as a star-schema, and can be one store within a broader polyglot-persistence strategy.

Example

A media company stores raw logs, clickstream, and content metadata as Iceberg tables on S3. Analysts query curated gold tables with Trino for BI, while data scientists train recommendation models on the same silver tables with Spark. Time-travel snapshots let them reproduce a model's exact training data, and open formats keep them free to swap compute engines.