Technologies tagged with "Managed Service"

The Lakehouse runtime catalog — until April 2026, BigLake metastore — is Google Cloud's fully managed, serverless implementation of the Apache Iceberg REST Catalog API, letting Spark, Flink, Hive, Trino, and BigQuery share the same Iceberg tables without duplicating files.

The Buf Schema Registry stores versioned Protobuf modules rather than individual topic schemas: pushes are linted and checked for breaking changes before they reach consumers, generated SDKs are published to native package managers, and a Confluent-compatible endpoint serves Kafka clients.

The AWS Glue Schema Registry is a serverless registry for streaming schemas — Avro, JSON Schema, and Protobuf — with eight compatibility modes, open-source serialisers, and integrations across MSK, Kinesis Data Streams, Managed Service for Apache Flink, and Lambda. AWS's documentation states it is free to use.

Soda is a data quality platform built on Soda Core, a Python library and CLI that runs checks and data contracts inside pipelines, with Soda Cloud for triage and collaboration and a hosted or self-hosted Runner for scheduled observability.

Elementary is an Apache-2.0 data observability tool built for dbt projects: a dbt package and CLI collect test results and run anomaly detection on freshness, volume, and custom metrics, with Elementary Cloud adding column-level lineage, ML monitors, and incident workflows.

Monte Carlo is a commercial data observability platform: it monitors tables automatically rather than waiting for hand-written tests, enriches alerts with lineage and root-cause context, and routes incidents through triage workflows.

Anomalo is a commercial data quality platform that applies machine learning to detect anomalies without hand-written rules, and can be deployed either as SaaS or entirely inside the customer's own VPC so that data never leaves their environment.

Datafold is a commercial platform built on value-level data diffing: it compares the output of a proposed change against production in CI, validates parity during warehouse migrations, and monitors metrics for deviation, deployable as SaaS or inside the customer's own cloud.

Collibra is a commercial data intelligence platform whose centre of gravity is governance: stewardship workflows, policies, and a business glossary, with a data catalog, lineage, quality and observability, a data marketplace, and AI governance built around them.

Alation is a commercial data catalog and data intelligence platform combining search and discovery, lineage, stewardship, and data quality, deployable either as the Alation Cloud Service or on the customer's own infrastructure.

Atlan is a commercial metadata platform covering discovery, lineage, glossaries, contracts, and policies across 80-plus sources, delivered as SaaS with a self-deployed runtime that keeps metadata extraction inside the customer's own network.

Microsoft Purview combines data governance — a Data Map that scans and classifies assets and a Unified Catalog above it — with data security and compliance solutions, billed through Azure meters for capacity, scanning, and governed assets.

Knowledge Catalog — previously Dataplex Universal Catalog, and the replacement for the retired Data Catalog — harvests metadata from BigQuery, Vertex AI, Pub/Sub, Bigtable, Cloud SQL, and AlloyDB into entries described by aspects, with natural-language search, lineage, and glossaries over the result.

Amazon DataZone is AWS's data management service for cataloguing, discovering, sharing, and governing data across AWS, on-premises, and third-party sources, organised around projects that publish assets and subscription requests that are approved through a governed workflow.

Kestra is an open-source orchestration platform whose workflows are declared in YAML rather than written in a programming language: an event-driven, language-agnostic engine with a large plugin catalogue, a JVM runtime backed by Postgres or MySQL, and Enterprise and Cloud editions above the Apache-2.0 core.

Temporal is an MIT-licensed durable execution platform: workflows are ordinary code in one of eight SDK languages, and the service persists every step so a function can keep running across crashes, restarts, and deploys for minutes or for months.

AWS Step Functions is a serverless orchestrator built on state machines: each step is a state that calls an AWS service or an external worker, execution history is retained and visually debuggable, and the bill is per state transition rather than per server.

Flyte is an LF AI & Data project for building production data and ML pipelines: tasks and workflows are Python functions with strongly typed interfaces, each execution is versioned and cacheable, and the whole thing runs on Kubernetes with a Go control plane underneath.

Windmill is an open-source workflow engine and developer platform written in Rust: individual scripts in TypeScript, Python, Go, SQL, Bash and more become reusable runnables, which compose into flows through a low-code builder or YAML, and into internal apps on top.

Power BI is Microsoft's business intelligence platform — free Windows authoring in Power BI Desktop, publishing and sharing through the Power BI service at a per-user price, and an on-premises Report Server for organisations that cannot publish to the cloud.

Tableau is Salesforce's business intelligence platform, licensed by role — Creator, Explorer, Viewer — and deployable either as the hosted Tableau Cloud or as Tableau Server on the customer's own Windows or Linux machines.

Looker is Google Cloud's business intelligence platform, built around LookML — a version-controlled modelling language in which metrics and joins are defined once and every dashboard, explore, and API call is generated from that definition rather than from ad-hoc SQL.

Amazon Quick Sight — until October 2025, Amazon QuickSight — is AWS's serverless business intelligence service, billed per user or per session, with SPICE, its in-memory engine, caching data for dashboards that must stay fast without hammering the source.

Sigma is a cloud analytics platform whose interface is a spreadsheet: business users pivot, filter, and add formulas against live warehouse tables, with input tables writing values back into the warehouse rather than into a local file.

Lightdash is an open-source BI platform that builds its semantic layer directly from a dbt project — dimensions and metrics are declared in the dbt YAML an analytics team already maintains, and every chart and dashboard is generated from those definitions.

Evidence is an MIT-licensed framework that builds business intelligence from plain files: SQL queries and charts are written inside markdown, and the project compiles to a static website that can be deployed anywhere a static site can.

Rill is an Apache-2.0 BI tool distributed as a single Go binary: metrics and dashboards are defined in SQL and YAML in a local project, served by an embedded DuckDB or a connected ClickHouse, and deployed to Rill Cloud when they are ready to share.

SQLMesh is a data transformation framework from Tobiko Data that runs SQL and Python models with a plan-and-apply workflow, column-level lineage, unit tests, and virtual data environments that let teams preview changes without rebuilding tables.

Dataform is a service for developing, testing, version-controlling, and scheduling SQL transformation workflows in BigQuery, built on an Apache-2.0 meta-language and offered on Google Cloud at no charge beyond the BigQuery it runs.

Coalesce is a commercial transformation platform that builds warehouse pipelines from reusable metadata-driven nodes and templates rather than hand-written SQL, with column-level lineage, cataloging, and quality checks in one managed service.

Bruin is an open-source data pipeline tool written in Go that combines ingestion, SQL and Python transformations, automated quality checks, and column-level lineage in a single CLI, with a managed cloud layer above it.

Presto is a distributed SQL query engine for federated analytics, governed by the Presto Foundation under the Linux Foundation, whose members include Meta, IBM, Uber, and Nvidia — the original codebase that PrestoSQL forked from before renaming itself Trino.

Polars is a DataFrame library written in Rust with a Python interface, built around a query optimiser and Arrow memory rather than row-by-row execution, and designed as a single-node engine that handles data most teams would otherwise put on a cluster.

Amazon Athena is a serverless interactive query service that runs SQL directly against data in Amazon S3, with no infrastructure to provision and a bill computed from the data each query scans.

Starburst is the commercial platform built on Trino, sold as Starburst Enterprise for self-managed clusters and Starburst Galaxy as a fully managed service, adding governance, performance work, and a supported Iceberg lakehouse on top of the open-source engine.

Dremio is a SQL engine for the lakehouse that federates queries across Iceberg, Delta Lake, databases, and warehouses, and accelerates them with materialised structures it calls Reflections — available as a free Community edition, a self-managed Enterprise edition, and a managed cloud service.