Open-source data integration with a large connector catalog
Ingestion
Airbyte is a data integration platform that syncs data from APIs, databases, and files into warehouses and lakes, available self-hosted or as a managed cloud service.
SQL-first transformation workflow for analytics engineering
Processing
dbt is a transformation framework that compiles modular, version-controlled SQL models and runs them inside the warehouse, with testing, documentation, and lineage built in.
In-process analytical database for local and embedded analytics
Processing
DuckDB is an open-source, in-process OLAP database — an embeddable columnar SQL engine that queries Parquet, CSV, and dataframes with no server to run.
Apache Airflow is the widely adopted open-source orchestrator where pipelines are Python-defined DAGs, with a large provider ecosystem and multiple managed offerings.