dlt
dlt is an open-source Python library for the extract-and-load stage, inferring schemas, normalising nested data, and handling incremental loading — installed with pip and run wherever Python already runs.
LicenseOpen source (permissive)· Apache-2.0
DeploymentSelf-hostedManaged
PricingFreeSubscription
Operational complexityLow
WorkloadBatch
LanguagePython
Use it when
- You want extract-and-load as code your team owns, not a platform to deploy or a subscription to manage.
- Pipelines must run where Python already runs: an Airflow task, a serverless function, CI, a laptop.
- Sources are REST APIs, SQL databases, or object storage, and you want schemas inferred, nested data normalised, and incremental state handled for you.
- Schema evolution needs governing: contracts can evolve, freeze, or discard changes per table or column.
Think twice when
- You expect maintained connectors for hundreds of SaaS sources; with dlt you write and own that code, where Airbyte or Fivetran ship it.
- The team prefers configuring pipelines in a UI over writing Python.
- You need built-in scheduling or monitoring; the library has none, so an orchestrator or dltHub provides it.
How it runs
pip install dlt, write a Python file, run it anywhere Python runs; there is no service to operate. Scheduling comes from whatever already runs your code (Airflow, Dagster, Prefect, or cron), and dltHub is the managed layer adding runtime, scheduling, and observability, billed in credits per runtime hour.
Details
CompareHow dlt answers the questions Batch ELT turns on.
Batch ELT
| How it works | |
| Authoring model | Python library: pip install and a Python file |
| Incremental sync | Cursor state per resource; append, replace or merge, including SCD2 |
| Sync frequency | Whatever your scheduler runs; dltHub adds scheduling and backfill |
| Schema drift | Evolves by default; contracts can freeze or discard, per table or column |
| Running it | |
| Scheduling | None in the library; run it from Airflow, Dagster, Prefect or dltHub |
| Connections | |
| Source coverage | REST APIs, SQL databases, object storage; SaaS templates in Python |
| Destinations | Warehouses, SQL databases, object storage with Delta or Iceberg, vector DBs |
| Custom connectors | Python: writing a source is the normal path |
| Transformations | None in the load; dbt runner, SQL client or Python after loading |
| Cost | |
| Billing unit | Free library; dltHub credits: one credit per runtime hour |
Alternatives to dlt
