Administration and Governance

    Who this is for:

    Architecture / Concept Overview: Administration and Governance

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% graph TD classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED Account[Account] --> WS1[Workspace: Production] Account --> WS2[Workspace: Development] Account --> WS3[Workspace: Staging] Account --> Metastore[Unity Catalog Metastore] Metastore --> WS1 Metastore --> WS2 Metastore --> WS3 Account --> Billing[Billing & Quotas] Account --> Identity[Identity Federation] Account:::governance WS1:::serving WS2:::processing WS3:::ingestion Metastore:::storage Billing:::source Identity:::source

    *Databricks account hierarchy: a single account governs multiple workspaces, shared metastores, billing, and identity federation.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% flowchart LR classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED Admin[Account Admin] --> Policies[Compute Policies] Policies --> Clusters[Cluster Creation] Clusters --> Usage[Usage Tracking] Usage --> SystemTables[System Tables] SystemTables --> Dashboards[Cost Dashboards] Dashboards --> Alerts[Budget Alerts] Admin:::governance Policies:::processing Clusters:::ingestion Usage:::source SystemTables:::storage Dashboards:::serving Alerts:::source

    *Administration workflow: policies govern compute creation, usage flows into system tables, and dashboards surface cost insights.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% graph TD classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED SCIM[SCIM Provisioning] --> Groups[Groups] Groups --> WsAssign[Workspace Assignment] Groups --> UCPerms[Unity Catalog Grants] Groups --> ComputePerms[Compute Permissions] WsAssign --> Users[Users] UCPerms --> Data[Data Access] ComputePerms --> Resources[Resource Access] SCIM:::ingestion Groups:::processing WsAssign:::storage UCPerms:::governance ComputePerms:::serving Users:::source Data:::storage Resources:::source

    *Identity governance: SCIM-provisioned groups cascade permissions to workspaces, data assets, and compute resources.*

    Key Terms

    Prerequisites and Setup

    • A Databricks account (Premium or Enterprise tier for full governance features)
    • Account admin role for billing, identity, and workspace management
    • At least one workspace deployed in your target cloud region
    • Identity provider configured for SCIM provisioning (Entra ID, Okta, or OneLogin)
    • Cloud provider IAM roles for cross-account access and storage credentials

    Step-by-Step Implementation

      Configuration Reference

      Administration and Governance configuration options
      SettingScopeValuesDefault
      Max WorkspacesAccountVaries by tierTier-dependent
      SCIM ProvisioningAccountEnabled / DisabledDisabled
      Compute PolicyWorkspaceCustom JSONUnrestricted
      Auto-terminationCluster10-120 minutes120 minutes
      System TablesAccountEnabled / DisabledEnabled (Premium+)
      Budget AlertsAccountCustom thresholdsNot configured
      Token LifetimeWorkspace1-730 days90 days
      Workspace AssignmentAccountPer-group / Per-userAll users

      Monitoring, Cost, and Security Considerations

      Common Pitfalls and Recommended Patterns

        Frequently Asked Questions