Favicon of Delta Lake

Delta Lake

Delta Lake is an open-source table format providing ACID transactions, schema enforcement, and time travel over Parquet files, closely integrated with Spark and Databricks.

LicenseOpen source (permissive)· Apache-2.0
DeploymentSelf-hostedManaged
PricingFree
Operational complexityMedium
LanguageScala, Java, Rust, Python

Use it when

  • Your platform is Spark- or Databricks-centric; Delta is the default format there and the integration is the deepest available.
  • You need ACID transactions, schema enforcement, upserts, and time travel over plain Parquet with no separate catalog service.
  • Downstream consumers read the Change Data Feed rather than rescanning tables.
  • Update-heavy tables benefit from deletion vectors, which avoid rewriting whole files on merge.

Think twice when

  • Writes come mostly from engines other than Spark; Trino, Flink, and the standalone readers are real but less complete than the Spark path.
  • Multiple clusters write to one table on S3, which needs a DynamoDB-backed LogStore or coordinated commits.
  • Renaming or dropping columns is routine; both require column mapping to be enabled first.
  • The rest of your stack has standardized on Iceberg; ecosystem alignment is usually the deciding factor between the two.

How it runs

Delta is a library in the engine plus a transaction log (_delta_log) stored inside the table path, so a catalog is optional. Data is Parquet on object storage; commits append to the log with optimistic concurrency. You run OPTIMIZE to compact and VACUUM to expire files; log checkpoints are automatic. Databricks operates all of this as a managed service, and UniForm can write Iceberg and Hudi metadata alongside for outside readers.

Details

Compare

How Delta Lake answers the questions Table Formats turns on.

Table Formats
How it works
CatalogOptional: _delta_log in the table path
Update modelCopy-on-write; merge-on-read with deletion vectors
PartitioningHive-style columns, or liquid clustering with CLUSTER BY, up to four keys
Schema evolutionAdd and reorder freely; rename and drop need column mapping enabled
Concurrent writersOptimistic log commits; multi-cluster S3 needs a DynamoDB LogStore or coordinated commits
IndexesFile stats for data skipping, Z-order and liquid clustering; bloom filters on Databricks
Change feedChange Data Feed, written to _change_data
Running it
MaintenanceOPTIMIZE to compact, VACUUM to expire files; log checkpoints are automatic
Connections
Engine supportSpark-first; Trino, Flink and standalone readers
Cross-format readsUniForm writes Iceberg and Hudi metadata alongside, GA in 4.0

Share:

Alternatives to Delta Lake

Favicon

 

  
  
Favicon

 

  
  
Favicon

 

  
  

Used in architectures