# Apache Flink

> Stateful stream processing at scale

Apache Flink is an open-source framework for stateful computation over unbounded and bounded data streams, with event-time processing, exactly-once state consistency, and CDC ingestion via Flink CDC.

Website: https://flink.apache.org

## Use it when

- Correct low-latency results over large state are the requirement: event time, watermarks, and exactly-once state consistency are the reference implementation here.
- The job mixes paradigms; DataStream code, Table/SQL, and batch-as-bounded-stream share one engine.
- CDC belongs in the processor; Flink CDC snapshots and streams MySQL, PostgreSQL, MongoDB, and more into the same job that transforms them.
- You need ecosystem depth: Kafka, Pulsar, Kinesis, Iceberg, and Paimon connectors, plus managed offerings from every major cloud and vendor.

## Think twice when

- The team is small and jobs are simple; a JobManager-and-TaskManagers cluster with checkpoints and savepoints is real operational weight (ops complexity here is High) against SQL-first engines like Arroyo or RisingWave.
- Rescaling must be online; changing parallelism means restoring from a checkpoint or savepoint.
- Results must be queryable in place; Flink emits to sinks and is not a serving layer.

## How it runs

Self-host a cluster (JobManager plus TaskManagers) on Kubernetes or YARN, with state in HashMap or RocksDB backends checkpointed to durable storage. Applications use DataStream (Java/Scala), PyFlink, or Flink SQL. Managed Flink is sold by Confluent, Ververica, AWS, and others.

## Details

### General

| Attribute | Value |
| --- | --- |
| License | Open source (permissive) |
| SPDX identifier | Apache-2.0 |
| Deployment | Self-hosted, Managed |
| Workload | Streaming, Batch |
| Operational complexity | High |
| Pricing | Free, Subscription |
| Language | Java, Scala, Python |
| Repository | https://github.com/apache/flink |
| Documentation | https://nightlies.apache.org/flink/flink-docs-stable/ |
| Stars | 26.3k |

### Change Data Capture

| Attribute | Value |
| --- | --- |
| Sources | MySQL, PostgreSQL, Oracle, MongoDB, SQL Server, Db2 |
| Source requirements | ROW binlog on MySQL, logical decoding on Postgres, per connector |
| What you run | A Flink cluster: JobManager and TaskManagers |
| Initial snapshot | Lock-free incremental snapshot, chunked and resumable |
| End-to-end lag | Sub-second; changes flow into the same job that processes them |
| Delivery guarantee | Exactly-once |
| Schema drift | Schema evolution modes: evolve, try_evolve, lenient, ignore, exception |
| Transformations | Full Flink processing in the same job; YAML transform for pipelines |
| Destinations | Any Flink sink: Kafka, warehouses, table formats |
| Billing unit | None for the project; managed Flink billed by the vendor |

### Stream Processing

| Attribute | Value |
| --- | --- |
| Processing model | Record-at-a-time streaming; batch is a bounded stream |
| APIs | DataStream, Table/SQL, ProcessFunction |
| What you run | A Flink cluster: JobManager and TaskManagers |
| Connectors | Kafka, Pulsar, Kinesis, JDBC, filesystems, Iceberg and Paimon |
| Windowing | Tumbling, sliding, session, global |
| Event time and lateness | Event time with watermarks; allowed lateness and side outputs |
| State backend | HashMap, RocksDB, ForSt (experimental, disaggregated) |
| Delivery guarantee | Exactly-once |
| Reading results | Through sinks; queryable state exists but is deprecated |
| Scaling | Rescale from checkpoint; adaptive scheduler and reactive mode |

## Capabilities

- [Change Data Capture](https://matca.io/capabilities/change-data-capture)
- [Stream Processing](https://matca.io/capabilities/stream-processing)

---

Source: https://matca.io/apache-flink
Last updated: 2026-08-31T12:09:44.410Z