Apache ORC
Apache ORC is a columnar file format built for the Hadoop and Hive ecosystem: per-column min/max statistics and bloom filters for skipping data, support for ACID transactions and snapshot isolation, and native handling of Hive's compound types.
LicenseOpen source (permissive)· Apache-2.0
DeploymentSelf-hostedManaged
PricingFree
Operational complexityLow
WorkloadBatchInteractive
LanguageJava, C++
Use it when
- The estate is Hive-lineage (Cloudera, older Hadoop clusters); ORC is Hive's native format and the only one Hive ACID tables support.
- Readers should skip aggressively: file, stripe, and 10,000-row indexes with min/max statistics and bloom filters are mature and well tuned.
- Column-level security belongs in the file: AES encryption per column with masking is specified in the format, which no peer offers.
- You need Hive's full SQL type system, including unions and both timestamp flavors.
Think twice when
- You are building a new lakehouse; Parquet is the default every table format and engine assumes, and there is little reason to prefer ORC there.
- DuckDB or Arrow-native tools are central: DuckDB does not read ORC directly, and Arrow interop is a full decode.
- Tables run thousands of columns wide; the Protobuf footer is deserialized whole, the cost Nimble was built to escape.
- You expect the format to keep gaining features; it is carefully maintained rather than expanding.
How it runs
A library inside the engine with two independent implementations, Java and C++; files sit on HDFS or object storage and there is nothing to operate. Hive, Spark, Trino, Flink, Impala, and Iceberg read it, most with predicate pushdown. ACID tables add base and delta files with compaction driven through the Hive metastore.
Details
CompareHow Apache ORC answers the questions File Formats turns on.
File Formats
| How it works | |
| Adoption and maturity | The definition of maintained rather than expanding. An ASF top-level project whose format arrived in Hive 0.11 in 2013, with five release lines patched in parallel (2.3.1 on 16 July 2026 and 1.9.9 eight days earlier on 8 July 2026, after 2.2.2, 2.1.4, 2.0.7 and 1.9.8 all landed in January 2026. Someone is still shipping fixes for a line that started years ago. The project's adopters page names Facebook as an early adopter with more than 300 PB, LinkedIn, Trino, Vertica and Timber alongside eleven Apache projects. The 769 stars understate all of this) 516 forks and 21 open issues are the better signal for a format the Hadoop generation standardised on and nobody stars twice |
| Encodings and compression | A fixed, well-chosen set, and fixed is the operative word: the encodings live in the specification, so adding one is a specification change (precisely the constraint Vortex and Nimble exist to escape. Integers get RLEv1 or RLEv2, the latter switching between short-repeat, direct, patched-base for outlier-heavy data and delta for monotonic runs; strings get dictionary or direct chosen per column by cardinality. Underneath, generic compression is None, Zlib, Snappy, LZO, LZ4 or Zstd, applied in independent chunks with a three-byte header so a reader seeks into a stream without decompressing what precedes it. What no other row here has is column-level encryption, added in 1.6 and specified rather than bolted on: AES/CTR at 128 or 256 bits, chosen because it seeds identical data differently, needs no padding and still lets a reader seek into a stream; a random local key per encrypted column, wrapped by Hadoop KMS, Ranger, AWS, GCP or Azure; and masking) nullify by default, redact to constants, or sha256, with the file storing both variants, encrypted-unmasked for key holders and unencrypted-masked for everyone else, so an unauthorised reader gets masked values rather than an error |
| Indexes | The mature answer the newer formats are measured against, at three levels. File-level and stripe-level statistics sit in the footer, and within each stripe a row index covers every 10,000 rows, each entry carrying column statistics (value and null counts, min and max on most primitives, sums on numerics) plus the positions needed to seek to that row group's start. Bloom filters have been available since Hive 1.2 in two encodings, the legacy BLOOM_FILTER and BLOOM_FILTER_UTF8, hashing numerics with Thomas Wang's function and strings with Murmur3. Search arguments push predicates down to skip whole files, stripes or row groups. Set beside the challengers this is the middle position: more than Vortex, which has no bloom filters at all, and less than Nimble's sorted and hash indexes |
| Random access | Bounded, and the boundary is the row group. The row index stores positions for the start of each 10,000-row group, so a reader seeks there and decodes forward, a point lookup costs a row group rather than a file, which was a real advance over reading a stripe and is two orders of magnitude away from what the newer formats claim. There is no key index, no hash index and no per-row addressing in the format. ORC's actual answer to "fetch me this row" was never in the file layout but in the ACID row-id triple above it, which is a different mechanism for a different question |
| Wide schemas | The row where this format is the weak one, and the evidence is a neighbouring row's design document rather than a criticism invented here: Nimble was built to use "Flatbuffers instead of thrift/protobuf to more efficiently access large metadata sections", naming Parquet and ORC as the formats it replaces. The cost is structural, the file footer carries statistics for every column, every stripe footer carries stream and encoding entries per column, and it is all Protobuf that a reader deserialises whole before touching any data. At tens or hundreds of columns this is invisible. At the thousands-wide feature tables the challengers target, the footer becomes the performance problem |
| Nested data | The richest type system of the four, and the one place where the oldest format is the most complete. Struct, list, map and union as compound types (union being the one neither Parquet, Vortex nor Nimble models) with everything nullable including the compound types themselves. The scalar set is a SQL type system rather than an ML one: char and varchar beside string, decimal, date, and both timestamp and timestamp with local time zone, which the documentation recommends because it fixes an instant rather than a wall-clock reading. Files are self-describing and need no external schema |
| ACID primitives | The only row on this page that answers anything but "none", and the reason the column exists. ORC puts transaction support in the file: every row gets an automatically assigned identity as the triple of original transaction, bucket and row id, guaranteed unique; inserts, updates and deletes are recorded as base and delta files, merged by minor compaction and rewritten by major compaction; and the reader is given the list of committed transactions by the Hive metastore when a query starts, so it sees a consistent snapshot. Hive's own documentation states the consequence ("Only ORC file format is supported in this first release") and explains that any format with an explicit or implicit row id could work, but the integration was only ever done for this one. It is not OLTP, and says so: millions of rows per transaction, not millions of transactions an hour |
| Spec versions | Two format versions in thirteen years and an unfinished third, which is a different kind of stability from either challenger. ORCv0 shipped with Hive 0.11 and ORCv1 with Hive 0.12, readers detect which they are given, and writers can deliberately emit the older one so a mixed cluster keeps reading, PyArrow surfaces exactly that as file_version 0.11 or 0.12. ORCv2 has been public as an evolving draft that "should only be used for developers on the project", carrying a to-do list of RLEv3, stripelets for asynchronous I/O, dictionary encoding for floats and decimals, and the removal of RLEv1 and the non-UTF8 bloom filters. Meanwhile the real feature work lands inside v1 without a version bump, column encryption in 1.6 being the clearest case. Extremely safe to write, and a decade of evidence that changing the encoding set of a specification-defined format is the thing that does not happen |
| Connections | |
| Engine support | The strongest answer on the page and the reason anyone still writes ORC: Hive, where it originated and where it is the native format; Spark with column projection, predicate pushdown and a vectorized reader; Trino; Flink's Table API; Impala reading through the C++ library; Druid ingestion; Pig and MapReduce; and Iceberg, which takes it as one of three data file formats beside Avro and Parquet. Two independent implementations, Java and C++, and Arrow ships a reader and writer over the latter. The gap worth naming is at the other end of the page: DuckDB's own list of readable data sources runs CSV, JSON, Parquet, Delta Lake, Iceberg, Lance and Vortex, and does not include ORC, the standing request is answered with "go through Arrow". The thirteen-year-old format lost that race to a 0.8x one |
| Arrow interop | Real, and conversion-based rather than zero-copy. Arrow's C++ implementation ships an ORC reader and writer, and PyArrow exposes it as pyarrow.orc, read_table and write_table, ORCFile and ORCWriter, per-stripe reads, column selection, and a file_version option of 0.11 or 0.12 that surfaces the format's two versions directly in the API. Reading is a full decode into Arrow arrays, which is the cost every format designed before Arrow pays, and exactly the step the newer rows avoid by handing compressed arrays to the engine intact |
Alternatives to Apache ORC
