Skip to main content

Medallion Architecture

Medallion architecture refines lakehouse data through bronze, silver, and gold layers for progressive quality and reuse. It brings order and lineage to data lakes at the cost of duplicated storage and hops.

Type
Data
When to Use
Lakehouse Pipelines, Progressive Data Refinement, Multi Consumer Data

Medallion architecture (also called multi-hop) organizes data in a lakehouse into successive quality tiers named after medals: bronze, silver, and gold. Data flows in one direction, improving in structure, quality, and business value at each hop. The pattern brings order to a data lake that might otherwise become a swamp.

How It Works

  • Bronze (raw): ingested data is landed as-is from source systems — files, streams, database extracts — with minimal transformation, preserving full history and lineage. Bronze is the immutable system of record for what arrived.
  • Silver (cleaned/conformed): bronze data is validated, deduplicated, type-cast, joined, and conformed to a consistent schema. Silver tables are queryable, trustworthy, and represent business entities.
  • Gold (curated/aggregated): silver data is aggregated and modeled for specific consumption — dashboards, ML features, or department marts — often as star schemas or wide denormalized tables.

Each layer is typically stored as ACID table format (Delta Lake, Apache Iceberg, Hudi) on object storage, and pipelines (Spark, dbt, Databricks) move data forward. Reprocessing a downstream layer is always possible from the layer above.

When to Use It

Use medallion when building analytics or ML pipelines on a lakehouse where raw data must be preserved, multiple teams consume the same data at different refinement levels, and data quality must improve incrementally. It is a default organizing principle for modern Databricks and Iceberg-based platforms.

Trade-offs

The layered approach duplicates storage (the same data exists in three forms) and adds pipeline hops, increasing latency and cost for some use cases. The boundaries between silver and gold are conventions, not rules, and teams often disagree on what belongs where. Over-engineering small datasets into three layers adds needless complexity. Governance, naming, and lineage discipline are required to keep the layers meaningful.

Related Patterns

Medallion is the canonical organization for a data-lakehouse; gold-layer marts are frequently modeled as a star-schema, and the refinement flow echoes the layering of lambda-architecture.

Example

A retailer lands raw point-of-sale files in a bronze Iceberg table. A silver job cleans, deduplicates, and conforms transactions into a unified sales table. A gold job aggregates sales into a daily store-by-product mart modeled as a star schema, which powers executive dashboards while data scientists build features from the silver layer.