Open table formats bring ACID transactions, schema evolution, and time travel to data lakes. Compare Apache Iceberg, Delta Lake, and Apache Hudi today.
by Databricks Staff
Open table formats are metadata layers that sit on top of data files in object storage, adding ACID transactions, schema evolution, and time travel to data stored in a data lake. Apache Iceberg, Delta Lake, and Apache Hudi are the three main open table formats in production use today, and each turns a collection of Parquet or ORC files into a table that behaves like a database: readers see consistent results, writers can update and delete rows safely, and every change is tracked so earlier versions remain queryable.
This overview explains how the main open table formats work, how they compare on ACID transaction support and schema evolution, and how they relate to the data lakehouse architecture — drawing on storage-layer innovations like catalog-coordinated transactions, row lineage, and unified metadata to show where Delta Lake and Apache Iceberg are converging.
A data lake is a centralized repository built on low-cost object storage — Amazon S3, Azure Data Lake Storage, or Google Cloud Storage — that holds structured, semi-structured, and unstructured data in raw, native form. Organizations adopted data lakes because object storage scales cheaply and separates storage from compute, letting any query engine read the same data. Object storage was never built to guarantee consistency, though: it has no native concept of a table, schema, or transaction.
