Assumed context

Data team of 2–6 engineers; no dedicated platform team.

Cloud-deployed, AWS as the concrete example; picks note portability where it matters.

Batch-first; freshness target ≈ hourly, with an optional CDC path for operational tables.

Managed services preferred where cheap; OSS-permissive bias for anything load-bearing.

Analytics/BI workloads; no ML feature-serving requirements.

Out of scope: ML training/feature stores, sub-second serving layers, data mesh organizational patterns, multi-region DR.

Build a Lakehouse

Warehouse-grade analytics on open table formats over object storage, without warehouse lock-in.

Last reviewed: Jul 28, 2026

Assumed context

Data team of 2–6 engineers; no dedicated platform team.
Cloud-deployed, AWS as the concrete example; picks note portability where it matters.
Batch-first; freshness target ≈ hourly, with an optional CDC path for operational tables.
Managed services preferred where cheap; OSS-permissive bias for anything load-bearing.
Analytics/BI workloads; no ML feature-serving requirements.

Out of scope: ML training/feature stores, sub-second serving layers, data mesh organizational patterns, multi-region DR.

Pipeline flow

↓ raw files → Object storage

↓ change events

↓

↓ table metadata↓ tables → Transformation

↓ modeled tables

↓ SQL

Build a Lakehouse

Warehouse-grade analytics on open table formats over object storage, without warehouse lock-in.

Assumed context

Pipeline flow

Cross-cutting layers

Build a Lakehouse

Warehouse-grade analytics on open table formats over object storage, without warehouse lock-in.

Assumed context

Pipeline flow

Cross-cutting layers

Build a Lakehouse

Warehouse-grade analytics on open table formats over object storage, without warehouse lock-in.

Assumed context

Pipeline flow

Cross-cutting layers

Build a Lakehouse

Warehouse-grade analytics on open table formats over object storage, without warehouse lock-in.

Assumed context

Pipeline flow

Cross-cutting layers

1Batch ingestionAirbyte1 alternative

2Change data captureOptionalAirbyte1 alternative

3Object storageAmazon S31 alternative

4Table formatApache Iceberg1 alternative

5Table catalogAWS Glue Data Catalog1 alternative

6Transformationdbt1 alternative

7Query engineTrino1 alternative

8BI & explorationMetabase1 alternative

▤File formatApache Parquet1 alternativeApplies over 1 · 2 · 3

▤OrchestrationOptionalDagster2 alternativesApplies over 1 · 2 · 6

▤Data qualityOptionalGreat ExpectationsApplies over 6