Open-source data integration with a large connector catalog
Ingestion
Airbyte is a data integration platform that syncs data from APIs, databases, and files into warehouses and lakes, available self-hosted or as a managed cloud service.
Debezium is an open-source CDC platform that streams row-level changes from databases such as PostgreSQL, MySQL, and MongoDB, typically via Kafka Connect.
Column-oriented OLAP database for real-time analytics
Storage
ClickHouse is an open-source columnar database designed for low-latency analytical queries over large volumes, available self-hosted or as ClickHouse Cloud.
Distributed SQL query engine for federated analytics
Processing
Trino is an open-source distributed SQL engine that queries data where it lives — object storage, lakehouse table formats, and operational databases — through a single interface.
Apache Kafka is an open-source distributed log for high-throughput, durable event streaming, with a large ecosystem including Kafka Connect and Kafka Streams.
Kafka-compatible streaming platform in a single binary
Streaming
Redpanda is a Kafka API-compatible streaming platform written in C++, shipped as a single binary without ZooKeeper or a JVM, available self-hosted or as a managed cloud.
Apache Airflow is the widely adopted open-source orchestrator where pipelines are Python-defined DAGs, with a large provider ecosystem and multiple managed offerings.
DataHub is an open-source metadata platform providing search, discovery, and end-to-end lineage across warehouses, pipelines, and BI tools via push-based ingestion.
Unified metadata platform for discovery, lineage, and quality
Governance
OpenMetadata is an open-source metadata platform combining catalog, lineage, data quality tests, and collaboration on a single schema-first foundation with many connectors.