# Apache Avro

> Row-oriented serialization with first-class schema evolution

Apache Avro is a compact, schema-based, row-oriented serialization format — the default encoding for Kafka pipelines and schema-registry workflows.

Website: https://avro.apache.org/

## Use it when

- Producers and consumers upgrade independently on Kafka topics; reader and writer schemas resolve by a specified algorithm a registry can enforce.
- Every schema registry supports it: it is the original default at Confluent, and equally at Karapace, Glue, and Apicurio.
- Pipelines must read other teams' data without code generation; a GenericRecord decodes against a schema fetched at runtime.
- Raw zones ingest records in motion: container files carry the writer's schema and are splittable at sync markers.

## Think twice when

- The workload is scan-heavy analytics at rest; columnar Parquet is the complement, not the competitor, and usually the destination.
- Payloads must be readable without infrastructure; a bare Avro datum is undecodable without its schema, unlike protobuf's taggable bytes or plain JSON.
- Your types originate in gRPC services; reusing existing protobuf definitions usually outweighs Avro's more flexible evolution.
- You were counting on its RPC; it is specified but effectively unused.

## How it runs

Libraries in Java, Python, C#, C, C++, and Rust; schemas are JSON documents, so registries can store, diff, and validate them directly. Data travels as compact binary with a schema fingerprint, or in container files with the full schema in the header and optional deflate, snappy, or zstandard codecs. Nothing to operate beyond the registry you likely already run.

## Details

### General

| Attribute | Value |
| --- | --- |
| License | Open source (permissive) |
| SPDX identifier | Apache-2.0 |
| Deployment | Self-hosted, Managed |
| Workload | Streaming, Batch |
| Operational complexity | Low |
| Pricing | Free |
| Language | Java, C#, C, C++, Python |
| Repository | https://github.com/apache/avro |
| Documentation | https://avro.apache.org/docs/ |
| Stars | 3.3k |

### Serialization Formats

| Attribute | Value |
| --- | --- |
| Encoding | Two encodings from one schema, and a container around them. The binary encoding is the dense one and it is dense because it carries nothing but values (integers zig-zagged and varint-packed, strings and bytes length-prefixed, records simply their fields concatenated in schema order, enums as a zero-based position. The JSON encoding is the debuggable one, tagging union branches by type name. Around either sits the object container file: magic bytes, a metadata map holding the writer's schema and an optional codec, a random 16-byte sync marker, then blocks of objects) with null and deflate required and bzip2, snappy, xz and zstandard optional. For a single message there is also a framed form, two marker bytes and an 8-byte CRC-64-AVRO fingerprint of the schema ahead of the datum |
| Random access | Two levels with opposite answers, which is why this row is not the flat "none" the neighbouring Thrift row gives. Inside a datum there is nothing to seek by and less to work with than Thrift has: with no field tags and no lengths on records, nothing in the bytes says where one value ends, so position is knowable only by walking the schema. A field the reader does not want is still ignored rather than jumped, the specification is categorical that "a schema must always be used in order to read Avro data correctly". At the file level the design is the reverse: a random 16-byte sync marker written between blocks lets a reader open a container file at an arbitrary offset, scan forward to the next marker and start decoding there. That is what makes Avro files splittable across mappers, and it is the property the Hadoop generation actually bought them for |
| Self-describing | At the file level yes, at the datum level not at all, and the gap between those is the format's whole design. The specification is blunt about the bytes (binary encoding "does not include field names, self-contained information about the types of individual bytes, nor field or record separators") so a bare Avro datum without its schema is undecodable, where a Thrift or protobuf message can at least be walked by its type tags. What Avro does instead is carry the full writer's schema in the container file's header, so a file is more self-describing than any tagged format: not just the shape but the field names, the documentation strings and the defaults. For single messages the fingerprint form points at a schema held elsewhere, which is the pattern every Kafka registry implements |
| Schema evolution | The reason to choose Avro, and the only row here where evolution is a specified algorithm rather than a set of conventions. Resolution matches the writer's schema against the reader's: fields are paired by name, a field the writer has and the reader does not is ignored, a field the reader has and the writer does not takes its declared default, and without a default it is an error. Numeric promotion is defined (int to long, float or double, long to float or double, float to double) as is string to bytes and back, and aliases let a renamed record or field still resolve. Enums resolve unknown symbols to a reader-side default. Because the rules are in the specification rather than in each team's habits, a registry can check a proposed schema against the previous one and refuse it, which is exactly what the schema-registry leaf does |
| Registry support | Universal, and the format the registries were built for. Confluent's own documentation puts it plainly: Protobuf and JSON Schema are supported "along with Avro, the original default format". Karapace holds the identical three, AWS Glue the same three with Avro pinned at 1.11.4, and Apicurio takes it among nine artifact types; only Buf's registry, which is Protobuf and nothing else, does not. This is the positive half of the column the Thrift row answers with a flat none, and it is the practical reason a Kafka platform that has not already chosen otherwise ends up here |
| Generated code | Optional, and this is the capability's clearest genuine advantage rather than a preference. The Java guide gives the mechanism in a sentence: data is always stored with its schema, so "we can always read a serialized item regardless of whether we know the schema ahead of time", which "allows us to perform serialization and deserialization without code generation". A GenericRecord reads a schema the program was never compiled against (which is what a pipeline handling other teams' topics actually needs) while generated SpecificRecord classes are there when compile-time types and speed matter more. Every other row in this capability treats the compiler as mandatory; this one treats it as a convenience |
| Type system | Eight primitives (null, boolean, int, long, float, double, bytes, string) and six complex types: record, enum, array, map with string keys, union, and fixed. Two of those carry weight the other rows do not. null is a type rather than a modifier, so optionality is expressed as a union of null with something, which is why unions appear everywhere in real Avro schemas. And logical types annotate the primitives with meaning (decimal over bytes or fixed, uuid, date, time and timestamp at millisecond and microsecond precision, local-timestamp variants, and duration) so a reader that does not know an annotation still reads the underlying value correctly. Schemas are themselves JSON, which is what lets a registry store, diff and validate them without a special parser |
| RPC | Specified and largely unused, which is a different answer from either having RPC or not. The specification defines protocols as JSON documents carrying named messages with request parameters, a response schema and an error union, a one-way form for null responses, a wire exchange of metadata plus name plus parameters, and a handshake in which client and server exchange MD5 hashes of their protocols and the server answers BOTH, CLIENT or NONE, stateless transports repeating it every request, stateful ones until it succeeds. HTTP is defined as a transport with content type avro/binary. It is a complete design that the ecosystem passed over: where Thrift's RPC is the reason people adopt Thrift, Avro's is a section of the specification most Avro users never read |
| Spec versions | Stable in the way that matters and versioned with the library rather than apart from it: the specification ships per release (1.12.0 is the one published as current) and old data keeps reading, which is the guarantee a format whose whole purpose is reader/writer skew has to make. Additions have been additive and infrequent, logical types and the single-object fingerprint form being the substantive ones, and nothing in the encoding has been revised out from under existing files. The visible weakness is documentation drift rather than format churn: the site still presents 1.12.0 as current while 1.12.1 and 1.12.2 have shipped, and the version-by-version documentation index makes that easy to miss |
| Adoption and maturity | The default for record data in streaming pipelines, and the project claims exactly that ("the leading serialization format for record data, and first choice for streaming data pipelines". The repository dates to May 2009, carries about 3,300 stars and 1,770 forks, and is still moving: 1.12.0 in August 2024, 1.11.5 and 1.12.1 in September 2025, and 1.12.2 tagged on 7 August 2026 and published five days later. Rust support has moved out to a separate apache/avro-rs repository, itself active. The honest qualification is that its position is inherited as much as won) Hadoop made it the record format and Confluent made it the Kafka default, and the challenge now comes from Protobuf's reach rather than from anything technical in this column |

## Capabilities

- [Serialization Formats](https://matca.io/capabilities/serialization-format)

---

Source: https://matca.io/apache-avro
Last updated: 2026-08-31T11:40:47.089Z