Delta Lake is an open-source table format providing ACID transactions, schema enforcement, and time travel over Parquet files, closely integrated with Spark and Databricks.
Delta is a library in the engine plus a transaction log (_delta_log) stored inside the table path, so a catalog is optional. Data is Parquet on object storage; commits append to the log with optimistic concurrency. You run OPTIMIZE to compact and VACUUM to expire files; log checkpoints are automatic. Databricks operates all of this as a managed service, and UniForm can write Iceberg and Hudi metadata alongside for outside readers.
How Delta Lake answers the questions Table Formats turns on.
| How it works | |
| Catalog | Optional: _delta_log in the table path |
| Update model | Copy-on-write; merge-on-read with deletion vectors |
| Partitioning | Hive-style columns, or liquid clustering with CLUSTER BY, up to four keys |
| Schema evolution | Add and reorder freely; rename and drop need column mapping enabled |
| Concurrent writers | Optimistic log commits; multi-cluster S3 needs a DynamoDB LogStore or coordinated commits |
| Indexes | File stats for data skipping, Z-order and liquid clustering; bloom filters on Databricks |
| Change feed | Change Data Feed, written to _change_data |
| Running it | |
| Maintenance | OPTIMIZE to compact, VACUUM to expire files; log checkpoints are automatic |
| Connections | |
| Engine support | Spark-first; Trino, Flink and standalone readers |
| Cross-format reads | UniForm writes Iceberg and Hudi metadata alongside, GA in 4.0 |
vs Delta Lake: Java
vs Delta Lake: Java, Python, Rust, Go
vs Delta Lake: Java