Open Table Formats
Data architecture technologies for formats that layer ACID tables, schema evolution, and time travel over files in object storage.
Transactional lake format built around upserts and incremental reads
Table Formats
Apache Hudi is an open lakehouse format that pioneered transactional data lakes, offering record-level upserts and deletes, incremental queries, and a choice between copy-on-write and merge-on-read table types.
Lake format with LSM-backed streaming updates and changelogs
Table Formats
Apache Paimon is a lake format unifying batch and streaming, using an LSM tree to absorb high-frequency updates and emitting changelogs so downstream jobs can consume what changed rather than rescanning.
Lakehouse format keeping all metadata in a SQL database
Table Formats
DuckLake is an integrated data lake and catalog format from the DuckDB Foundation that stores table metadata in an ACID SQL database rather than in files, with data in Parquet on object storage.
Lakehouse format for multimodal AI with fast random access
Table Formats
Lance is an open lakehouse format for multimodal AI data, claiming 100x faster random access than Parquet, with vector and full-text indexes in the format specification and versioning built in.
Open table format for large analytic datasets
Table Formats
Apache Iceberg is an open table format bringing ACID transactions, schema evolution, and time travel to data lake tables, with broad engine support.
Open storage framework with ACID transactions for data lakes
Table Formats
Delta Lake is an open-source table format providing ACID transactions, schema enforcement, and time travel over Parquet files, closely integrated with Spark and Databricks.