Apache Hudi

Use it when

• You are applying record-level upserts and deletes to lake tables continuously, such as CDC ingestion from operational databases.
• Downstream pipelines should consume only what changed, using incremental queries or the CDC view instead of full scans.
• The workload is write-heavy or streaming and merge-on-read fits: appends land fast and compaction is deferred.
• Key-based writes benefit from its indexes (bloom, bucket, record-level) to locate affected files quickly.

Think twice when

• Your engines need to write from outside Spark or Flink; Trino, Presto, and Hive only read Hudi tables.
• You want to minimize table operations: compaction, clustering, cleaning, and archival are yours to configure and run.
• Multiple writers must commit concurrently; optimistic concurrency needs an external lock provider except in the non-blocking MoR case.
• The data is slow-changing batch output, where Iceberg or Delta Lake involves fewer moving parts.

How it runs

Hudi is a library inside the engine, not a server. Tables live on object storage with the transaction timeline in the table directory, optionally synced to a metastore or catalog. You choose copy-on-write or merge-on-read per table and schedule maintenance inline, semi-async, or as separate jobs. Managed platforms from the ecosystem around the project run those services for you, and Apache XTable can translate the metadata to Iceberg or Delta in place.