Databricks Terraform Provider: Provisioning Workspaces as Code
Who this is for:
Architecture / Concept Overview: Databricks Terraform Provider: Provisioning Workspaces as Code
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
flowchart LR
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
HCL[Terraform HCL Config]:::source
TF[Terraform Engine]:::ingestion
PROVIDER[Databricks Provider]:::processing
API[Databricks REST API]:::storage
INFRA[Workspace Infrastructure]:::serving
STATE[State Backend]:::governance
HCL --> TF --> PROVIDER --> API --> INFRA
TF --> STATE
*Terraform translates HCL configurations into API calls via the Databricks provider, maintaining state for drift detection.*
%%{init: {"theme":"base","themeVariables":{"background":"#0B0E14","primaryTextColor":"#E0E6ED","lineColor":"#5D6470","darkMode":true,"primaryColor":"#2E4A4A","secondaryColor":"#374151","secondaryTextColor":"#E0E6ED","tertiaryColor":"#111827","tertiaryTextColor":"#E0E6ED","edgeLabelBackground":"#1f2937"}}}%%
graph TD
classDef source fill:#3F4B59,stroke:#9CA3AF,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef ingestion fill:#5A4B36,stroke:#C9A86B,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef processing fill:#535072,stroke:#8E82B4,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef storage fill:#2E4A4A,stroke:#5FAFA8,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef serving fill:#3D5550,stroke:#6BB7AA,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
classDef governance fill:#5A3F52,stroke:#C28BB0,stroke-width:2px,rx:8,ry:8,color:#E0E6ED
TF_RES[Terraform Resources]:::source
WS[Workspace Provisioning]:::ingestion
COMPUTE[Compute Configuration]:::processing
SECURITY[Security & Access]:::governance
UC[Unity Catalog]:::storage
DATA[Data Resources]:::serving
TF_RES --> WS
TF_RES --> COMPUTE
TF_RES --> SECURITY
TF_RES --> UC
TF_RES --> DATA
WS --> MWS[databricks_mws_workspaces]:::ingestion
COMPUTE --> CLUSTER[databricks_cluster]:::processing
COMPUTE --> POLICY[databricks_cluster_policy]:::processing
SECURITY --> GROUPS[databricks_group]:::governance
SECURITY --> PERMS[databricks_permissions]:::governance
UC --> CATALOG[databricks_catalog]:::storage
UC --> SCHEMA[databricks_schema]:::storage
*The provider supports resources across workspace provisioning, compute, security, and Unity Catalog domains.*
Key Terms
Prerequisites and Setup
- Terraform 1.0+ installed
- Databricks account with admin or workspace admin permissions
- Service principal with appropriate scopes for CI/CD
- Remote state backend (S3, Azure Blob, GCS) for team collaboration
- Cloud provider credentials for workspace provisioning (if creating new workspaces)
Step-by-Step Implementation
Configuration Reference
| Resource | Purpose | Key Arguments |
|---|---|---|
databricks_cluster | Managed cluster | spark_version, node_type_id, autoscale |
databricks_cluster_policy | Cluster guardrails | definition (JSON policy document) |
databricks_job | Scheduled job | task, schedule, cluster |
databricks_catalog | Unity Catalog catalog | name, storage_root |
databricks_schema | Catalog schema | catalog_name, name |
databricks_grants | Permission grants | catalog/schema/table, grant blocks |
databricks_secret_scope | Secret storage | name, backend_type |
databricks_mws_workspaces | Workspace provisioning | workspace_name, cloud_resource_container |