Setting Up Unity Catalog in Your Workspace
Who this is for:
Architecture / Concept Overview: Setting Up Unity Catalog in Your Workspace
Setting up Unity Catalog involves configuring resources at both the account level and the workspace level.
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
ADMIN[Account Admin] -->|Creates| SC[Storage Credential<br/>IAM Role / Managed Identity]
ADMIN -->|Creates| META[Metastore<br/>+ Managed Storage]
SC --> META
ADMIN -->|Assigns| WS[Workspace<br/>to Metastore]
META --> WS
ADMIN -->|Configures| SCIM[SCIM Provisioning<br/>IdP → Databricks]
WS --> CAT[Default Catalog: main]
ADMIN:::source
SC:::ingestion
META:::governance
WS:::processing
SCIM:::governance
CAT:::storage
*Figure 1 — Unity Catalog setup flow: account admin creates storage credentials and metastore, assigns workspaces, and configures identity federation.*
The cloud-specific storage credential architecture differs by provider.
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
SC[Storage Credential] --> AWS[AWS<br/>IAM Role with trust policy]
SC --> AZURE[Azure<br/>Managed Identity or Service Principal]
SC --> GCP[GCP<br/>Service Account]
AWS --> S3[S3 Bucket<br/>Managed storage root]
AZURE --> ADLS[ADLS Gen2 Container<br/>Managed storage root]
GCP --> GCS[GCS Bucket<br/>Managed storage root]
SC:::governance
AWS:::processing
AZURE:::processing
GCP:::processing
S3:::storage
ADLS:::storage
GCS:::storage
*Figure 2 — Storage credential types and their corresponding cloud storage backends.*
Key Terms
Prerequisites and Setup
- Databricks account admin credentials
- A cloud storage bucket/container for managed storage (created before metastore setup)
- An IAM role (AWS), managed identity (Azure), or service account (GCP) with read/write access to the storage location
- An identity provider (Azure AD, Okta, OneLogin) for SCIM provisioning
Step-by-Step Implementation
Configuration Reference
| Setting | Scope | Recommended Value |
|---|---|---|
| Metastore region | Account | Same region as your workspaces |
| Managed storage root | Metastore | Dedicated bucket/container per metastore |
| Default catalog name | Workspace | main (or a team-specific catalog) |
| SCIM provisioning | Account | Enabled with automatic sync from IdP |
| Metastore admin | Account | A group (e.g., platform-admins) rather than an individual |
| Storage credential rotation | Account | Rotate IAM credentials per your security policy |