Customer-Managed Keys for Encryption

    Who this is for:

    Architecture / Concept Overview: Customer-Managed Keys for Encryption

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% graph TD classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED Customer[Customer KMS] --> MasterKey[CMK Master Key] MasterKey --> |"Encrypts DEKs"| DEK1[DEK: Managed Services] MasterKey --> |"Encrypts DEKs"| DEK2[DEK: Workspace Storage] MasterKey --> |"Encrypts DEKs"| DEK3[DEK: Cluster Volumes] DEK1 --> Notebooks[Notebooks & Queries] DEK2 --> DBFS[DBFS Root & Tables] DEK3 --> EBS[Local SSDs / EBS] Customer:::governance MasterKey:::processing DEK1:::storage DEK2:::storage DEK3:::storage Notebooks:::ingestion DBFS:::serving EBS:::source

    *CMK envelope encryption: the customer's master key encrypts workspace-specific data encryption keys, which in turn encrypt actual data.*

    %%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%% flowchart LR classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED CreateKey[Create CMK] --> SetPolicy[Set Key Policy] SetPolicy --> RegisterKey[Register with Databricks] RegisterKey --> AssignWorkspace[Assign to Workspace] AssignWorkspace --> VerifyEncryption[Verify Encryption] VerifyEncryption --> ScheduleRotation[Schedule Rotation] CreateKey:::source SetPolicy:::ingestion RegisterKey:::processing AssignWorkspace:::storage VerifyEncryption:::serving ScheduleRotation:::governance

    *CMK lifecycle: from key creation through workspace assignment to ongoing rotation management.*

    Key Terms

    Prerequisites and Setup

    • Databricks Premium or Enterprise tier account
    • Cloud KMS access (AWS KMS, Azure Key Vault, or GCP Cloud KMS)
    • Account admin permissions in Databricks
    • Cross-account IAM role or service principal for Databricks to access your KMS
    • Understanding of key rotation and disaster recovery requirements

    Step-by-Step Implementation

      Configuration Reference

      Customer-Managed Keys for Encryption configuration options
      SettingAWSAzureGCP
      Key ServiceAWS KMSAzure Key VaultCloud KMS
      Key TypeSymmetric (AES-256)RSA-HSM or AESSymmetric (AES-256)
      Use CasesMANAGED_SERVICES, STORAGEMANAGED_SERVICES, STORAGEMANAGED_SERVICES, STORAGE
      Cross-RegionMulti-Region KeysGeo-ReplicationGlobal Keys
      RotationAnnual (automatic)Configurable periodAnnual (automatic)
      RevocationDisable/Schedule DeleteDisable/PurgeDisable/Destroy
      HSM SupportCloudHSM backendManaged HSMCloud HSM

      Monitoring, Cost, and Security Considerations

      Common Pitfalls and Recommended Patterns

        Frequently Asked Questions