Flyte
Flyte is an LF AI & Data project for building production data and ML pipelines: tasks and workflows are Python functions with strongly typed interfaces, each execution is versioned and cacheable, and the whole thing runs on Kubernetes with a Go control plane underneath.
LicenseOpen source (permissive)· Apache-2.0
DeploymentSelf-hostedManaged
PricingFreeSubscription
Operational complexityHigh
WorkloadBatch
LanguageGo
Use it when
- Reproducibility is the feature: typed task interfaces are checked before anything runs, executions are versioned, and unchanged tasks are skipped through caching.
- ML and data pipelines share one platform on Kubernetes, with Spark and Ray operator plugins.
- Typed Python that produces containers for you beats writing container specs by hand: the Argo trade, one level up.
Think twice when
- There is no Kubernetes cluster or platform team; the control plane is heavier than Argo's and much heavier than Prefect's.
- Generation risk matters: Flyte 2.0 shipped in late 2025 with the 1.x line behind it, and the distributed Flyte 2 backend is still maturing.
- Vendor concentration is a concern: an LF AI & Data project, but developed principally by Union.ai, which also serves its documentation.
- Tasks must be authored in something other than Python; the Java SDK is archived.
How it runs
Self-hosted on Kubernetes: FlytePropeller creates a pod per task and deletes it after, connectors hand tasks to external services without holding a pod, and triggers cover cron, fixed rate, and artifact events. Typed I/O offloads large values to blob storage. Union.ai sells the managed platform, billed per action plus allocated compute.
Details
CompareHow Flyte answers the questions Workflow Orchestration turns on.
Workflow Orchestration
| How it works | |
| Authoring model | Typed Python: tasks are functions whose inputs and outputs are checked before the run |
| Unit of work | A task: an async Python function under @env.task, grouped by a TaskEnvironment carrying the image, resources and defaults. Flyte 2 dropped the static @workflow DAG: composition is ordinary Python, tasks awaiting tasks, and fan-out is asyncio.gather |
| Passing data | Typed function signatures checked against their annotations, with the io types (File, Dir, DataFrame) offloading large values to blob storage and passing a reference rather than the bytes |
| Triggers | A Trigger attached to a task, automated by Cron with a five-field expression and a timezone, FixedRate in minutes, or OnArtifact, firing whenever a named artifact is produced; runs otherwise start from the CLI, the SDK or the API |
| Delivery guarantee | Retries per task; caching skips one whose inputs and version are unchanged |
| Caching and reruns | cache="auto" hashes the function body, image and code bundle into a version, so editing the function invalidates the entry; "override" pins a version you set and "disable" turns it off. ignored_inputs drops inputs from the key, salt namespaces it, and serialize makes concurrent identical calls wait on one result |
| Running it | |
| Where tasks run | A pod per task, created and deleted by FlytePropeller; operator plugins for Spark and Ray, and Web API tasks that start no pod |
| Connections | |
| Task languages | Python and nothing else: Flyte 2 ships one SDK, `flyte` on PyPI, and flytekit-java was archived in April 2026. Other languages run inside the task's own container image, or behind a connector |
| Connectors | flyte.connectors, the mechanism Flyte 1 called agents, a registry of async connectors that hand a task to an external service and poll it rather than holding a pod for the duration; anything not covered is a container image |
| Cost | |
| Billing unit | Free to self-host: Flyte 1 in full, and Flyte 2 through a single-binary chart still at v0.2.0 while its distributed backend is "coming soon". Union.ai, the vendor, bills per action at $0.0075, an action being one invocation of a task, trace or condition, with retries not counted and duration irrelevant, from $950 a month that is itself usage credit, plus allocated compute by the second |
Alternatives to Flyte
