Delta Lake Open Protocol and Apache Iceberg Compatibility
Who this is for:
Architecture / Concept Overview: Delta Lake Open Protocol and Apache Iceberg Compatibility
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
DT[Delta Table] -->|Write via Delta Protocol| LOG[Delta Transaction Log]
LOG -->|UniForm generates| ICE[Iceberg Metadata]
LOG -->|UniForm generates| HUDI[Hudi Metadata]
ICE -->|Read| TRINO[Trino / Presto]
ICE -->|Read| SNOWFLAKE[Snowflake]
LOG -->|Read| SPARK[Spark / Flink]
LOG -->|Read| DUCKDB[DuckDB / Polars]
DT:::storage
LOG:::governance
ICE:::processing
HUDI:::processing
TRINO:::serving
SNOWFLAKE:::serving
SPARK:::serving
DUCKDB:::serving
*UniForm maintains parallel metadata alongside the Delta log, enabling Iceberg and Hudi readers to access the same data without conversion.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
PROTOCOL[Delta Lake Protocol] --> ACTIONS[Actions]
PROTOCOL --> FEATURES[Protocol Features]
ACTIONS --> ADD[AddFile]
ACTIONS --> REMOVE[RemoveFile]
ACTIONS --> META[Metadata]
ACTIONS --> COMMIT[CommitInfo]
FEATURES --> CDF_F[Change Data Feed]
FEATURES --> DV_F[Deletion Vectors]
FEATURES --> CM_F[Column Mapping]
FEATURES --> LC_F[Liquid Clustering]
FEATURES --> UNIFORM_F[UniForm]
PROTOCOL:::governance
ACTIONS:::processing
FEATURES:::storage
ADD:::processing
REMOVE:::processing
META:::processing
COMMIT:::processing
CDF_F:::ingestion
DV_F:::ingestion
CM_F:::source
LC_F:::serving
UNIFORM_F:::serving
*The Delta protocol defines atomic actions (AddFile, RemoveFile) and versioned features that engines opt into via reader/writer protocol versions.*
Key Terms
Prerequisites and Setup
- Databricks Runtime 14.1 or later for UniForm support
- Unity Catalog with external storage configured
- Understanding of which downstream engines need to read your Delta tables
ALTER TABLEprivilege to enable UniForm on existing tables
Step-by-Step Implementation
Configuration Reference
| Property | Default | Description |
|---|---|---|
delta.universalFormat.enabledFormats | — | Comma-separated list: iceberg, hudi |
delta.columnMapping.mode | name | Required for UniForm compatibility |
delta.minReaderVersion | 3 | Required for UniForm (v3 minimum) |
delta.minWriterVersion | 7 | Required for UniForm (v7 minimum) |
delta.enableDeletionVectors | true | Recommended with UniForm |
delta.sharing.enabled | false | Enables Delta Sharing on the table |