Data Lakehouse

Structured data sits in the warehouse, while log data, documents, sensor data and datasets for AI live in other systems. Reporting and AI end up working on separate data sets that take real effort to reconcile.

A data lakehouse stores everything in open formats in cloud storage and provides tables on top with the properties of a data warehouse. Whether that fits the data and the goals is the first thing we check. What both worlds have in common, such as layers and history tracking, is covered on the data warehouse page and not repeated here.

Files, table layer, catalog

A lakehouse has three parts. At the bottom are files in cloud storage, usually in Parquet format. On top sits a table layer that delivers what plain files cannot: ACID transactions, schema management, changes to individual rows and access to earlier table states (time travel). The catalog records which tables exist and who may access them.

Different engines work on these tables: SQL queries for reports, Spark for data preparation and machine learning, streaming jobs for continuously arriving data. That is the real difference from a classic warehouse, where storage and query engine are tightly coupled.

When a lakehouse can do more than a warehouse

For mostly structured data and classic reporting, we recommend the warehouse. Building a lakehouse just for the name makes things more complicated, not better.

It pays off when the data looks different or is used differently. When very large volumes accumulate that are cheap to keep in object storage and grow independently of compute, for instance. Or when documents, images, logs, JSON and sensor data should be available next to the tables. Or when models are trained on the same data that reporting runs on, without creating copies, and AI applications use documents and tables as a knowledge base. Data that arrives continuously and should be queryable without detours points the same way.

The two aren’t mutually exclusive. Often a lakehouse for raw data and AI sits next to a warehouse for verified reporting, or the warehouse itself runs on open formats.

Delta, Iceberg or Hudi

Delta Lake originated around Databricks and is the foundation of Microsoft Fabric: all tables in OneLake are Delta tables. Apache Iceberg is read by most major platforms, though to different degrees. Apache Hudi is mostly used for frequent updates of individual records and for streaming.

The formats are converging. In OneLake, Microsoft offers a mechanism that lets the same table be read as Delta Lake and as Iceberg without copying the data, and other vendors also expose Delta tables to Iceberg readers. The decision therefore depends less on the format itself than on which tools need to read and write it. Features and version support differ by platform, and we check that before settling anything.

The catalog decides how open it really is

What matters is whether several engines can use the same catalog. The Iceberg REST Catalog has become the common interface. It is implemented by Apache Polaris, among others, which became a top-level project of the Apache Software Foundation in February 2026 and is backed by vendors such as Snowflake. Databricks brings Unity Catalog, and Google and AWS have catalogs of their own.

Open formats in your own storage reduce dependence on a single vendor. Switching platforms doesn’t mean reloading all the data. That is not complete protection. Access rules, classifications and lineage are often tied to a vendor’s catalog, some platforms read tables from other engines but only write to their own, and newer format features don’t arrive everywhere at the same speed. We therefore decide deliberately where openness matters and where a vendor’s features should take priority.

Since tables, files and models share one platform, the rules have to apply centrally too: access rights down to row and column level, classification of sensitive data, traceability of where data comes from, logging. They are best maintained in the catalog so that reports, notebooks and AI applications see the same rules. Which rules apply and who is responsible for them is set by data governance. On the lakehouse, we implement them technically.

How a lakehouse goes wrong

Usually the purpose is missing. If you only have structured data for reports, a warehouse gets you there faster. The second classic is the data lake that becomes unmanageable: without structure, ownership and a catalog, nobody finds what exists and nobody trusts the data.

Then there are costs and permissions. Unrestricted queries on large data volumes quickly consume a lot of compute, and access rules are hard to enforce after the fact. Every additional combination of format and engine also adds maintenance. And when reporting and AI share the same data, metrics have to be defined once and data quality checked, otherwise you just get more variants of the same number, faster.