AWS Glue Data Catalog
The AWS Glue Data Catalog is a fully managed, Hive-compatible metastore that resolves table definitions — including Iceberg tables — for engines across the AWS analytics stack.
LicenseCommercial
DeploymentManaged
PricingFreeSubscription
Operational complexityLow
WorkloadInteractiveBatch
Use it when
- The platform lives on AWS: Athena, EMR, Redshift, Glue ETL, and SageMaker resolve tables through it natively, with zero operations.
- Table upkeep should be managed: compaction, snapshot retention, and orphan-file deletion run as built-in optimizers billed by the DPU-hour.
- Fine-grained access matters: Lake Formation adds row- and column-level permissions and vends scoped, temporary S3 credentials.
- Schemas should be inferred from data already in S3; crawlers populate the catalog for Hive-style, Iceberg, Hudi, and Delta tables.
Think twice when
- Catalog portability is a requirement; the coupling to AWS is exactly what Polaris or Lakekeeper exists to avoid.
- You want git-style branches, tags, or cross-table commits; versioning stops at table versions and Iceberg snapshots (Nessie's territory).
- Metadata scale is unusual: pricing counts every table, version, partition, and index as a billable object past the free million.
- Engines run outside AWS and you would rather not authenticate everything with SigV4.
How it runs
Nothing to deploy: a serverless, regional AWS service. It speaks three protocols (the Glue API, a Hive Metastore-compatible surface, and an Iceberg REST endpoint) and can federate outward to remote catalogs such as Unity Catalog and Snowflake's. Optimizers run as managed jobs, optionally inside your VPC. Charged per metadata object stored and per request, with the first million of each free.
Details
CompareHow AWS Glue Data Catalog answers the questions Table Catalogs turns on.
Table Catalogs
| How it works | |
| Protocol | Three surfaces: the Glue API, a Hive Metastore-compatible one, and an Apache Iceberg REST endpoint authenticated with SigV4 that implements the specification's operations. Catalog federation runs the other way too, letting the Data Catalog front remote Iceberg catalogs including Databricks Unity Catalog and Snowflake's |
| What it catalogs | Tables and the things around them, enumerated precisely because AWS bills by them: a table, a table version, a partition, a partition index, statistics, a database or a catalog each count as one metadata object. No files, functions or models, this catalogs the lakehouse rather than everything a team owns |
| Branching and versioning | Table versions and Iceberg snapshots, and no branching, a table version is a first-class object AWS counts and bills, and snapshot retention governs how much history survives. There are no branches, tags or cross-table commits of the git-style kind |
| Running it | |
| Maintenance | Three optimisers, and this is the answer that sits beyond the spec, compaction in binpack, sort or Z-order strategies, snapshot retention with a configurable period and count, and orphan file deletion, each enabled per table or catalog-wide, optionally running inside your own VPC, billed at $0.44 a DPU-hour |
| What you run | Nothing: a serverless AWS service. Crawlers infer schemas from data already in place, and the optimisers run as managed jobs rather than as something anyone schedules |
| Connections | |
| Table formats | Hive-style tables natively, plus the three open formats (Iceberg, Hudi and Delta Lake) populated by crawlers or written by ETL jobs. A Hudi merge-on-read table lands as two Data Catalog tables where copy-on-write lands as one |
| Engine support | The AWS analytics stack by default (Athena, EMR, Redshift, Glue ETL and SageMaker Lakehouse) with the Iceberg REST endpoint opening it to anything carrying an Iceberg REST client, which in AWS's own worked examples includes open-source Spark, PyIceberg, Databricks and Salesforce Data 360 |
| Access | |
| Access control | Lake Formation over IAM: fine-grained permissions down to rows and columns, evaluated when a principal reaches a table, and extended across federated catalogs so a remote Unity Catalog or Snowflake table is authorised exactly as a local one is |
| Storage credentials | Lake Formation vends them: scoped, temporary credentials to the S3 data behind a table, so an engine never holds a long-lived cloud key, and the same mechanism carries fine-grained permissions through to federated tables |
| Cost | |
| Billing unit | Objects and requests, each with a million free, a million metadata objects stored at no charge and $1 per 100,000 a month above that, and the first million requests free. Optimisation is metered separately at $0.44 a DPU-hour, billed by the second with a one-minute minimum, and statistics generation at the same rate |
Alternatives to AWS Glue Data Catalog
