DataHub
DataHub is an open-source metadata platform providing search, discovery, and end-to-end lineage across warehouses, pipelines, and BI tools via push-based ingestion.
LicenseOpen source (permissive)· Apache-2.0
DeploymentSelf-hostedManagedHybrid
PricingFreeSubscription
Operational complexityHigh
WorkloadInteractiveBatch
LanguagePython, Java, TypeScript
Use it when
- Engineers need search, discovery, and end-to-end column-level lineage across warehouses, pipelines, and BI tools.
- Ingestion surface matters: close to a hundred connector modules, configured as YAML recipes and run on a schedule, from the UI, or from the CLI.
- Metadata should stay current by streaming change events rather than by nightly crawls.
- The model must be extensible: new aspects and entity types are schema files the storage, indexing, and GraphQL layers pick up.
Think twice when
- Operating it is the objection: a self-hosted deployment carries Kafka, Elasticsearch, and a relational store, and the ops complexity is real.
- You need native quality assertions or subscriptions in open source; both are DataHub Cloud features, where OpenMetadata ships tests in its free edition.
- The governance office is the buyer; Collibra-style stewardship workflow is not the design centre.
How it runs
Self-hosted on Kafka, Elasticsearch, and a database, or through DataHub Cloud, the managed offering. Metadata changes flow in as events by push or scheduled pull; search and lineage serve from the indexed graph; policies govern who may edit metadata rather than who may query sources.
Details
CompareHow DataHub answers the questions Data Catalogs turns on.
Data Catalogs
| How it works | |
| Metadata model | Schema-first in Pegasus PDL: an entity is a type plus a URN plus aspects, an aspect is the smallest atomic unit of write and is shared across entity types, and relationships are declared by annotation on aspect fields so the graph walks both ways. The whole model is stitched together by an entity-registry.yml read at boot |
| Lineage | End-to-end and column-level, assembled from connector metadata and from parsing the SQL the platforms already ran, with impact analysis walking the graph downstream; DataHub Cloud adds propagation along that lineage so a tag or term travels with the data |
| Search and discovery | Search across every entity type at once (datasets, charts, dashboards, pipelines, ML models) with domains, tags, glossary terms and ownership as the facets that narrow it, over an Elasticsearch index kept current by the ingestion stream |
| Business glossary | Terms and term groups are entities in the graph rather than labels on one, they carry their own ownership, documentation and relationships, and attach to entities and to individual columns |
| Quality checks | The open-core line runs through this one, the open-source edition ingests results computed elsewhere, while DataHub Cloud adds native assertions for freshness, volume, columns and custom SQL, plus Smart Assertions that learn what normal looks like from history and surface anomalies beside the asset in search |
| Custom metadata | The aspect model is the extension point, write a PDL schema for a new aspect, or a whole new entity type, and name it in entity-registry.yml, and the storage, indexing and GraphQL layers pick it up. Snapshot models were the old way and were deprecated in January 2022 |
| Connections | |
| Connectors | Close to a hundred modules under metadata-ingestion, counted from the tree (warehouses, lakes, BI tools, orchestrators, ETL and streaming platforms alike) configured as YAML recipes run on a schedule, from the UI, or from the CLI. The largest first-party ingestion surface in this capability |
| Access | |
| Policy and compliance | Over metadata, not data: platform policies grant privileges like managing users, viewing analytics or editing policies themselves, and metadata policies say who may change which aspect of which entity, both declared through the UI or GraphQL. Nothing here gates a query against the underlying source |
| Access requests | None in the open-source edition, and no request-and-grant flow in either, DataHub Cloud adds subscriptions and notifications so a change to an asset reaches the people who depend on it, which is the opposite direction of travel from an access request |
| Cost | |
| Billing unit | Free to self-host, and the paid edition publishes nothing, DataHub Cloud has no pricing page, so neither a figure nor a unit can be recorded. The same answer Collibra and Atlan give, from an open-core rather than a closed product |
Alternatives to DataHub
