Databricks-to-Databricks Secure Data Sharing
Who this is for:
Architecture / Concept Overview: Databricks-to-Databricks Secure Data Sharing
D2B sharing leverages Unity Catalog's metastore federation to create a zero-copy link between provider and consumer workspaces. The consumer sees shared assets as a read-only catalog.
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
A[Provider Workspace] -->|Create Share| B[Unity Catalog Share]
B -->|Grant| C[Recipient Record]
C -->|Metastore Link| D[Consumer Workspace]
D -->|Mount as Catalog| E[Shared Catalog]
E -->|Query| F[Provider Storage]
F -->|Governed by| G[Provider Permissions]
A:::source
B:::governance
C:::ingestion
D:::processing
E:::storage
F:::storage
G:::governance
*D2B sharing creates a metastore-level link so the consumer queries provider data in place, governed by provider-defined permissions.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
A[Provider Actions] --> B[Create Share]
A --> C[Add Tables / Views]
A --> D[Register Recipient]
A --> E[Grant Access]
F[Consumer Actions] --> G[Accept Share]
F --> H[Create Catalog from Share]
F --> I[Query Shared Data]
F --> J[Grant Local Permissions]
A:::source
B:::governance
C:::storage
D:::ingestion
E:::governance
F:::processing
G:::processing
H:::storage
I:::serving
J:::governance
*The provider creates and governs the share, while the consumer accepts it, mounts it as a catalog, and queries with local permissions layered on top.*
Key Terms
Prerequisites and Setup
- Both provider and consumer must have Databricks workspaces with Unity Catalog
- The consumer must share their metastore sharing identifier with the provider
- Metastore admin privileges on the provider side to create shares and recipients
- Metastore admin privileges on the consumer side to create catalogs from shares
Step-by-Step Implementation
Configuration Reference
| Parameter | Description | Default |
|---|---|---|
share.name | Unique share identifier in the provider's metastore | Required |
recipient.sharing_id | The consumer's metastore sharing identifier | Required |
share.auto_update | Include new partitions automatically | true |
share.history_data_sharing | Enable CDF for incremental reads | disabled |
catalog.name | Name for the shared catalog on the consumer side | Required |
catalog.provider_share | Reference to the provider's share | Required |
recipient.comment | Description of the recipient organization | Empty |