Data Sharing and Collaboration

    Who this is for:

    Architecture / Concept Overview: Data Sharing and Collaboration

    Data sharing on Databricks spans three patterns: Databricks-to-Databricks sharing for internal and partner exchange, Delta Sharing for open cross-platform distribution, and Marketplace for public or commercial data products.

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% flowchart LR classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED A[Data Provider] -->|Delta Sharing Protocol| B[Open Recipients] A -->|D2B Sharing| C[Databricks Recipients] A -->|Marketplace Listing| D[Marketplace Consumers] A -->|Clean Room| E[Joint Analysis Partners] B -->|Any Platform| F[Pandas / Spark / Power BI] C -->|Unity Catalog| G[Shared Catalog] D -->|Install| H[Consumer Workspace] A:::source B:::serving C:::processing D:::ingestion E:::governance F:::serving G:::storage H:::storage

    *Databricks supports four sharing patterns — open protocol, Databricks-to-Databricks, Marketplace, and Clean Rooms — each optimized for different collaboration scenarios.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% graph TD classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED A[Sharing Capabilities] --> B[Delta Sharing] A --> C[OpenSharing] A --> D[D2B Sharing] A --> E[Marketplace] A --> F[Clean Rooms] B --> G[Open Protocol / Cross-Platform] C --> H[Non-Databricks Recipients via Link] D --> I[Databricks-to-Databricks via Unity Catalog] E --> J[Discover & Publish Data Products] F --> K[Privacy-Safe Joint Computation] A:::processing B:::serving C:::serving D:::storage E:::ingestion F:::governance G:::source H:::source I:::storage J:::ingestion K:::governance

    *The Databricks sharing ecosystem includes open protocols, managed sharing, marketplace distribution, and privacy-preserving collaboration.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% flowchart LR classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED A[Provider Unity Catalog] -->|Create Share| B[Share Object] B -->|Add Assets| C[Tables / Views / Models / Notebooks] C -->|Grant to Recipient| D[Recipient Record] D -->|Credential Exchange| E[Consumer Access] E -->|Query| F[Shared Data] A:::governance B:::processing C:::storage D:::ingestion E:::serving F:::source

    *The sharing workflow follows a consistent pattern: create a share, add assets, grant to recipients, and consumers access the shared data.*

    Key Terms

    Prerequisites and Setup

    • A Databricks workspace with Unity Catalog enabled
    • Metastore admin or data sharing admin privileges
    • Tables or other assets in Unity Catalog that you want to share
    • For D2B sharing: the recipient workspace's sharing identifier
    • For Delta Sharing: the recipient organization's sharing credentials or activation link

    Step-by-Step Implementation

      Configuration Reference

      Data Sharing and Collaboration configuration options
      ParameterDescriptionDefault
      share.nameUnique share identifier within the metastoreRequired
      share.commentHuman-readable descriptionEmpty
      recipient.typeDATABRICKS or OPENInferred from creation method
      recipient.sharing_idThe metastore sharing identifier (D2B)Required for D2B
      recipient.token_lifetimeValidity period for open sharing tokens90 days
      share.auto_updateAutomatically include new partitionstrue
      share.history_data_sharing_statusShare Change Data Feed (CDF)disabled

      Monitoring, Cost, and Security Considerations

      Common Pitfalls and Recommended Patterns

        Frequently Asked Questions