Use it when

Several consumers need to read the same events independently, or replay history from a point in time.
You are moving CDC streams, application events, or telemetry at a volume where a broker that routes each message would not keep up.
Ordering within a key matters, since a partition preserves it.
Kafka Connect and Kafka Streams cover the integration and processing you need without a separate engine.

Think twice when

You need request/reply, per-message routing, or priority queues; a broker such as RabbitMQ fits that shape better.
Nobody on the team can run a cluster, and a managed service (Confluent Cloud, Amazon MSK) or a Kafka-compatible single binary (Redpanda) is not an option.
Your volume is small enough that a database table or a managed queue would do.

How it runs

Brokers in KRaft mode (ZooKeeper is gone in 4.0) hold a partitioned, replicated commit log; tiered storage can offload older segments to object storage. Scale by adding brokers and reassigning partitions. Kafka Connect runs as a separate cluster of workers for source and sink integration, and Kafka Streams is a library inside your application, not a server.

Details

Compare

How Apache Kafka answers the questions Event Streaming turns on.

Event Streaming

How it works

Protocol

Kafka: defines it

Consumption model

Replayable log with offsets; share groups add queues since 4.2

Storage

Replicated local disks; tiered storage to object store

Delivery guarantee

At-least-once by default; Exactly-once with transactions

Running it

What you run

Brokers in KRaft mode; ZooKeeper removed in 4.0

Scaling

Add brokers, then reassign partitions; tiered storage lightens it

Multi-region

MirrorMaker 2, asynchronous; no built-in active-active

Connections

Connectors

Kafka Connect: a separate cluster of workers

Transformations

Kafka Streams library and Connect SMTs; no server-side engine

Cost

Billing unit

None for the project; managed Kafka billed by the vendor

Alternatives in Event Streaming

View all technologies

Amazon Kinesis Data Streams

AWS-native shard-based streaming with on-demand capacity

vs Apache Kafka: Commercial · Managed · Serverless · Subscription · Operational complexity: Low

Amazon Kinesis Data Streams is AWS's managed event streaming service, organising data into shards with provisioned or on-demand capacity and integrating directly with the rest of the AWS analytics stack.

Apache Pulsar

Streaming platform separating serving from storage tiers

vs Apache Kafka: Java

Apache Pulsar is an open-source messaging and streaming platform that splits brokers from storage, keeping segments in Apache BookKeeper so capacity and serving scale independently, with multi-tenancy and geo-replication built in.

AutoMQ

Kafka with its storage layer rebuilt on object storage

vs Apache Kafka: Free · Subscription · Operational complexity: Medium · Java

AutoMQ keeps Apache Kafka's compute layer and replaces its storage with cloud object storage, claiming 100% Kafka API compatibility and substantially lower cost by removing multi-replica local disks and cross-zone transfer.